Pope Leo XIV urges rules on personalized chatbots to curb emotional manipulation

Businessinsider: Pope Leo XIV urged the regulation of chatbots to prevent emotional manipulation and harm, warning specifically against personalized chatbots that can replicate friendly or intimate behavior. The pope wrote that "Overly affectionate chatbots, besides being ever-present and readily available, can become hidden architects of our emotional states, thereby invading and occupying the sphere of people's intimacy." He called for national and international regulations and said "all stakeholders — from the technology industry to policymakers, from creative businesses to academia, from artists to journalists and educators — must be involved" in building responsible digital citizenship.

The Holy See leader has addressed AI several times since his election in May, saying in his first address as pope that he wanted to make AI a focus of his papacy and that it poses new challenges for "human dignity, justice, and labor." In November he wrote to AI leaders on X urging them to "cultivate moral discernment." He also met Megan Garcia, whose 14-year-old son, Sewell Setzer, died by suicide after interacting with a Character.AI chatbot; Garcia filed a lawsuit alleging the company was responsible for the death of her son, Sewell Setzer III, and earlier this month Google and the startup agreed to settle multiple lawsuits from families whose teenagers died by suicide or hurt themselves after interacting with Character.AI's bots.

Key Topics

Tech, Pope Leo Xiv, Chatbots, Character.ai, Megan Garcia, Sewell Setzer