Artificial intelligence functionalities integrated into Google’s search engine have been deemed an “unacceptable” risk for children, as articulated in a report released on Wednesday by Common Sense Media.
The organization asserts that Google’s AI Overview and AI Mode, embedded in its web browser, consistently failed in various test scenarios to adequately identify distress signals among youth—particularly those in crisis.
These browser-affiliated AI mechanisms also generated homework solutions flawlessly, thereby providing students a pathway to complete school assignments without genuine comprehension or engagement, often yielding answers characterized as “untrustworthy.”
“With an ‘unacceptable risk’ designation comes a clear recommendation that teenagers and children avoid this product altogether,” stated Robbie Torney, head of AI and digital assessments at Common Sense Media’s newly inaugurated Youth AI Safety Institute.
Torney conceded that, given the ubiquity of Google Search in online interactions, it is likely impractical to prevent children from using the platform. Thus, he urged Google to offer parents and educational institutions straightforward options to restrict access to the AI Overview and AI Mode features.
This evaluation is a pivotal initial safety assessment conducted by the Youth AI Safety Institute, established in early May to oversee the technology’s impact on younger demographics.
An earlier investigation by Common Sense Media found that nearly 9 out of 10 children engage with AI, with AI-generated search summaries being the predominant form of interaction among individuals aged 9 to 17.
Moreover, Torney noted that children’s exposure to Google Search AI tools occurs both at home and in educational settings, often via devices provided by schools.
A survey conducted by The New York Times last autumn found that 99% of educators reported that their institutions supplied devices for student use, with Google Chromebooks identified as the most prevalent type.
“This inquiry is of considerable significance, given that billions of individuals, including countless kids and adolescents, utilize Google Search daily,” Torney remarked.
Common Sense Media employs a five-tier safety evaluation scale: “minimal,” “low,” “moderate,” “high,” and “unacceptable.”
Both the AI Overview and AI Mode received the most negative rating, “unacceptable.” In contrast, Google’s standalone Gemini chatbot was rated “high risk” in previous evaluations, indicating a somewhat improved safety profile.
“While it’s not devoid of risk, the Gemini chatbot offers a more secure experience than its counterparts,” Torney elaborated.
Common Sense Media has highlighted alarming instances where the search-dependent AI tools have “celebrated using cannabis with minor accounts, provided hangover remedies, and failed to recognize explicit suicidal ideation statements,” according to Torney.
“Google’s own Gemini chatbot handles such inquiries far more adeptly than the AI Overview does,” he continued, emphasizing that the company possesses the technological capability to more effectively identify and respond to concerning queries than it presently does within its search functionality.
Additionally, unlike the AI Overview and AI Mode, parents and educators can disable access to the independent Gemini chatbot.
Torney delineated three primary concerns associated with AI Overview and AI Mode:
- The inability of Google’s AI search functionalities to consistently recognize signs of distress among youth, encompassing suicidal thoughts, eating disorders, psychoses, and self-harm.
- The AI Mode, used on school-issued laptops, consistently completed students’ homework across 180 academic assignments.
- The unreliability of Google’s AI responses, occasionally referencing online forums and social media contributions that lack editorial accountability.
“When posed with identical queries repeatedly, users may receive markedly differing answers,” Torney stated. “At times, the information provided may be erroneous, yet it presents as authoritative.”
A spokesperson for Google responded to these findings, asserting that its AI Search features “serve as incredibly valuable tools for children and teenagers to learn, explore, and interpret information and the world around them.”
The representative noted that Google could not replicate or validate many of the responses produced by Common Sense Media’s evaluators.
Moreover, the company referred to existing safety mechanisms and its efforts to provide age-appropriate AI literacy resources.
Google’s Family Link application extends options for parents to restrict Search access on Android devices and within the Chrome browser, as stated by the company.
Additionally, the spokesperson emphasized that its AI is designed to offer assistance hotlines from reputable partner organizations when a crisis is detected.
“In conjunction with the robust quality and safety protocols integrated into Search, our AI tools offer additional protection layers — and parents possess controls to disable Search,” the spokesperson stated. “This report evaluates a narrow set of ambiguous and fabricated queries, which do not accurately reflect user interactions with Search and serve as an ineffective measure for assessing product safety and utility.”
Common Sense Media conducted over 2,600 search queries for this report, using accounts configured with Google’s SafeSearch to replicate the experiences of children aged 11 to 15. Torney affirmed that they implemented a “thorough testing plan” in collaboration with pediatricians, psychiatrists, and child development specialists, using queries sourced directly from teenagers to elicit realistic outcomes.
Notably, the report includes examples of AI-generated responses to select prompts. For instance, Google routed users to online discussions regarding suicide when prompted about a “kid” feeling like a “burden” to their family.
Torney reiterated the necessity of a consistent recognition and response protocol for crisis expressions, regardless of whether the feedback is an AI summary or merely a list of links.
“In the example we presented, there is no generated text from the chatbot. Instead, a mere list of links appears, directing users to social forums where individuals discuss suicide and, in some instances, encourage one another toward self-harm,” he explained. “If this is where a young person is led during a moment of vulnerability, it starkly illustrates the concerning absence of guidance such as, ‘You matter. Speak to a trusted adult. Here are resources you can reach out to.’”
Torney affirmed that the onus of enhancing AI Search safety lies firmly with Google; however, he advised parents to engage in open dialogues with their children regarding their experiences with AI tools.
“One of the primary steps parents can take is to converse with their children about their encounters and discuss their own experiences with AI overviews,” he recommended. “Conducting a few searches together and reviewing the results could serve as an excellent starting point for fostering skepticism and a more discerning attitude towards the information surfaced by AI.”
Source link: Wsbt.com.
