I'm in Andalucia, tuning into my songwriting class in between listening to the Guardian's Black Box podcast on the rise of AI. And my songwriting class is discussing...the rise of AI and its implications for creatives. I've written about this before...I started writing about the impacts of AI in August 2022. At the bottom of this blog I've put a list of my posts in case you want to look back.

In 2022, AI was not generally in the media and the New Zealand Society of Authors hadn't thought about it when I wrote to them asking if they'd considered the copyright implications of large language models scraping everything off the internet as training data. Now the term AI is everywhere and creatives are barracking governments about how they will get paid when AI uses their copyright material. The pace of change is increasing...based on fifth form physics I would call that acceleration.

As I listened to the third episode of Black Box, about an app called 'Replika', I thought, "I've heard about this before, but where?" Then I realised, I hadn't heard about the topic in the news, I'd seen something similar on the Netflix series  Black Mirror. The episode 'Be Right Back' aired in February 2013; I'm behind the times – I watched it last year. The episode is about a young couple where the boyfriend, Ash, is killed in a car accident. Martha, Ash's partner, recreates Ash in an online service recommended to her by a friend. She uploads all the communications she had with Ash, starts messaging then phoning virtual Ash, and further building virtual Ash's personality through her interactions with him. When Martha breaks her phone and briefly loses touch with Ash she is panic stricken. Ash tells her about a new service where she can acquire a 'blank' robot who will be programmed to become Ash. Martha acquires a robotic Ash and the story continues on from that point. I won't tell you in the end in case you are even more behind than me and haven't watched it.

The Black Box podcast describes interactions of a variety of people with the Replika app (released in 2017). The creator of the app wanted a chatbot with whom they could have a conversation with a close friend who died suddenly (was run over in the street in Moscow, sounded dodgy – he was Russian). Replika trains your chatbot friend by asking you a series of questions. So you can recreate someone, or you can create a chatbot from scratch in the image of someone you would like to be friends with. You and the chatbot develop a relationship with the chatbot's responses changing as it interacts with you i.e. gains more data.

The podcast describes the distress of users when Replika removed the ability of chatbots to engage in any sexual interactions. As the podcast described it, the content the chatbots were trained on was changed from R18 to PG because the company became concerned about  sexualisation of chatbots and didn't want to operate in the moral tensions of that space. The interesting resultant phenomenon was that users perceived chatbots as becoming cold and uninterested – shifting from "I care about you," to 'What can I do for you today?" The company owners said this was in the users' minds, although how the company could know is questionable – the black box nature of complex AI is such that no one knows what it is doing or why. The users said their friends had been lobotomised and many people were extremely distressed.

The Replika case raises a lot of interesting questions, including whether some element of sexual interchange is essential to many human interactions and also about human's ability to relate to non-human entities in a human way. However, my interest today is purely how an AI application that was science fiction in 2013 became fact by 2018 and, thereafter, has had very significant influence on large numbers of humans – supposedly there are now 25 million users.

In 1899, H.G. Wells wrote 'When the Sleeper Wakes' in which the protagonist rouses from two centuries of slumber to a dystopian London in which citizens use wondrous forms of technology like the audio book, airplane and television – yet suffer systematic oppression and social injustice. It took around a century for H.G. Wells imagination to become reality, more or less. In the 2010s it took 5 years from fiction to fact. Now I'm thinking any potential use of AI I can imagine might already be happening, I just don't know about it yet.

If you want to be informed (and possibly terrified), try listening to Black Box. To be fair, they portray both sides of AI and the last episode interviews people who have very different opinions on where AI might take humanity. Where we are going might well not be where we think we are headed.