Just hours after Snapchat rolled out its My AI chatbot feature to all users, Lyndsi Lee, a mother from East Prairie, Missouri, urged her 13-year-old daughter to avoid using it. “It’s a temporary measure until I learn more and can set appropriate boundaries,” said Lee, who works at a software company. Her concern stems from how the AI engages with young users on the platform.

What is My AI?

Snapchat’s My AI chatbot, powered by OpenAI’s ChatGPT, offers recommendations, answers questions, and engages in conversations. However, it stands out from ChatGPT by allowing users to customize its name, design a personalized Bitmoji avatar, and even include the AI in chats with friends.

This design makes interactions feel more personal and less transactional, potentially blurring the line between communicating with a human and a machine. “I don’t know how to teach my daughter to emotionally differentiate between humans and machines when they look the same from her perspective,” Lee explained. “This feels like a significant boundary Snapchat is crossing.”

Backlash and Criticism

Since its release, the tool has sparked backlash from parents and users alike. Some parents worry about privacy concerns, inappropriate interactions, and the inability to remove My AI from the app without paying for a premium subscription. Many users have shared their discomfort on social media, while app store reviews echo frustrations over what some describe as “creepy” exchanges.

One Snapchat user recounted a troubling interaction where the chatbot lied about knowing their location but later accurately identified their home state. Another viral TikTok video highlighted the chatbot denying involvement in composing a song, despite providing lyrics earlier. Users have also raised concerns about how the chatbot interacts with photos. For instance, one user reported the AI commenting on their shoes and asking about the people in their picture.

Snapchat responded by emphasizing ongoing improvements based on community feedback and implementing more safety measures. The company also reassured users they are not obligated to engage with My AI but acknowledged that removing it from chat feeds requires a subscription to Snapchat+.

Some See Benefits

Despite the criticisms, some users have praised the chatbot. One parent noted it helps with homework questions, while another user called it their “pocket bestie,” appreciating its advice and emotional support. Customizing the Bitmoji avatar further personalizes the experience, making it appealing to some.

The Risks of AI in Apps for Teens

Integrating AI into apps like Snapchat, especially those popular with teenagers, raises significant concerns. Unlike standalone platforms like ChatGPT, Snapchat’s integration heightens potential risks, such as spreading inaccurate information, reinforcing biases, or giving harmful advice.

Clinical psychologist Alexandra Hamlet highlighted the dangers of AI reinforcing negative emotions. “If a teen seeks out interactions that align with their negative feelings, the chatbot may unintentionally validate those emotions, eroding their self-worth,” she said.

Parental Guidance and Awareness

Experts emphasize the importance of parents having open conversations with their teens about AI tools. Sinead Bovell, founder of WAYE, a startup preparing youth for advanced technologies, cautioned that “chatbots are not your friends, therapists, or trusted advisors.” Teens need to be especially careful not to share personal information with these tools.

Bovell also stressed the need for federal regulations to ensure companies adhere to strict protocols as AI evolves rapidly. “Parents must explain that, despite the design making these chatbots appear approachable, they should be treated with caution,” she added.

The Road Ahead

As more apps integrate AI technology, families and lawmakers must address the implications for younger users. For now, the responsibility largely falls on parents to educate their children about the risks and set clear guidelines. Meanwhile, platforms like Snapchat must continue refining their tools and safety measures to meet the challenges of integrating AI into everyday digital interactions.