We have all grown up with movies and stories of the after-life. The word séance evokes in us a more or less similar image of a group of people sitting, holding hands in a dark room, with the fantastical Ouija board in front of them. A gust of wind blows the lone candle out, and the dial on the board starts moving, spelling out a message from ancestors who have been long gone.
That image may not be the one upcoming generations grow up with anymore. Software company AE Studio along with OpenAI, launched a particularly curious software in the middle of 2023. Searching “ghosts” and “AI” on any search engine now throws up this search result : SeanceAI. Once you sign in, it gives you the option to connect with a lost loved one, and informs you that connecting with dead historical figures is a feature that “will be coming soon”.
The Race to Profit from Grief
Futurism reports that SeanceAI designer Jarren Rocks “leans into” the ghoulishness of the app, and is thus not wary of their straightforward branding. "We're trying to make it sound as magical and as mystical as possible," Rocks told Futurism, saying the name is a call to attention over how advanced large language model (LLM) tech has gotten.
LLMs and the chatbots that have evolved as a result of them have gained huge traction in the recent past, and people are increasingly forming para-social relationships with them. Such models feed on human loneliness and alienation. Talking with your favorite fictional character may ease that for even a few minutes, and thus it is an outlet that people turn towards nowadays. However, if that sounds exploitative, then promising grieving individuals that an AI software can help connect them to lost loved ones is beyond manipulative.
AI-developers are in no way the first to try and profit off death and grief, as several models that apparently help people cope with their loss have risen and fallen throughout the years on the internet and beyond. From selling memorabilia of a dead public figure, to using devastating events as marketing strategies to artists on the internet who make sculptures from the ashes of dead relatives or cups from their bones, the grief industry is greedy and expansive.
GhostBots, Grief-Tech and Ethics
SeanceAI is not the first or last to try and optimize personal loss and thus their own profit. However, combining the ethical considerations of AI as a whole, along with promising a mystic experience through that very technology and also profiting off vulnerable people makes the model truly despisable. "It's essentially meant to be a short interaction that can provide a sense of closure. That's really where the main focus is here," said Rocks. "It's not meant to be something super long-term. In its current state, it's meant to provide a conversation for closure and emotional processing."
The AI software collects your data by processing your mobile number or email account as you have to log in to use the app. It then goes on to ask you details such as your name, place of birth, date of birth and other information which it uses to generate the answers once questions are prompted to it. Therefore, not only is this unsafe in terms of breach of privacy, taken from a desperate user, but it is also an extremely manipulative way to do so.
Apart from this particular software, there has been a concerning rise of “death-deep fakes”, which can bring a lost one to life on screen, and a crop of California based startups such as Replika, HereAfterAI and StoryFile which all promise to connect the living with the deceased or provide “grief-therapy” by essentially interacting with LLMs. LLMs that allow the dead to interact with guests at their funeral are also on the rise.
Death is a lucrative business, wherein the possibilities are immense and the customers are desperate, oftentimes. Some of these start-ups are cautious in their approach, while others, like Jarren Rock of SeanceAI, are quite aggressive, stating that "It's essentially meant to be a short interaction that can provide a sense of closure. That's really where the main focus is here."
Sidelining Grief : Whose Profit?
In the light of such developments, maybe there is a larger conversation to be had here. The Centre for Loss and Trauma blames the manner in which society forces individuals to ignore and be wary of grief their entire life for the manner in which we grow desperate in the face of loss, reaching out to any outlet that promises comfort, even though we may be getting actively scammed.
An user may write on SeanceAI - “Hey Dad, I am an engineer now.” The bot will reply, “Wow! That’s great, I am so proud of you, Riya.” This sweetness however, fades away soon, as the bot tends to get repetitive.
Late-capitalism will wretch everything out of the common man’s hand, even a grieving man. These start-ups will come up and die out, and new schemes will emerge, but maybe we are the ones who should be thinking of change, and a healthier relationship with grief as well as technology.
コメント