Edward Saatchi, CEO of Fable Studio and maker of Emmy-winning virtual reality experiences, participated in a thoughtful conversation about “virtual beings” at our recent GamesBeat Summit 2020 event.
I called the session “We are who we pretend to be,” after the moral of story in one of my favorite novels, Mother Night, by Kurt Vonnegut. The novel is about an American spy in World War II who does too good a job at his cover role of being a Nazi propagandist. The moral is: “We are who we pretend to be, so we must be careful about what we pretend to be.”
We chose to talk about the ethics and promise of virtual humans, or artificial people created for video games and other experiences. Saatchi is the organizer of The Virtual Beings Summit. His event explores what it means to create virtual beings, and it has been held a couple of times since mid-2019, and Saatchi is organizing a new event for June.
I did a rehearsal talk with Saatchi where we talked about other things as well, and I’ve included quotes from that conversation in this story, as well as quotes from the embedded video that aired at our event.
From Saatchi’s view, AI is the next great art form. It is fighting for its legitimacy now, just as virtual reality, video games, films, comics, and other things had to fight for their legitimacy in the past.
“As game developers start to explore natural language processing, computer vision, and synthetic speech, we could move away from the slightly repetitive releases we have had recently,” said Saatchi, who is trying to bring together game developers and AI technologists through his summit. “I think there’s a lot of opportunity in developers exploring machine learning and artificial intelligence as if it were an art form.”
Dark history
Above: A rogue Android kidnaps a girl in Detroit: Become Human.
I opened with a question about the usual dark vision associated with artificial people, going back to Mary Shelley’s Frankenstein. Today, we’ve got Black Mirror, the Terminator, Westworld, Detroit: Become Human, and more. I asked, “What are we so afraid of?”
We can see how fresh these fears are with Neon, a Samsung-backed spinoff that touted its artificial humans as assistants for people at CES 2020, the big tech trade show that took place in January. You could rent these people, who could become a doctor, chef, stewardess or some kind of replacement for a human.
It’s part of what Saatchi has called the “replacement fear narrative” that we have about artificial intelligence, or AI, replacing us. Westworld raises another “valid” fear about how AI could come to dominate humans, while another fear is loneliness, based on the worry that we could trick ourselves into having relationships with these beings out of loneliness, he said.
“There are quite a few movies and books, all with possible negative outcomes. That’s not what I believe, but it’s a pretty cool canon,” Saatchi said.
In Detroit: Become Human, there’s massive unemployment, caused by the economic impact of robots replacing people. It’s a video game where we see the psychological impact, where people can inflict their darker impulses on artificial beings that we tell themselves aren’t real people, Saatchi said.
Then there’s Lucy
By contrast, Saatchi’s team created the cartoonish virtual reality (and soon to be non-VR) character Lucy, from Wolves in the Walls, a VR experience inspired by a book by Neil Gaiman.
“For us, there’s a world where virtual beings could help us become better people,” Saatchi said. “Beings who aren’t selfish, who aren’t motivated by greed or envy, who are able to listen to us and validate us and see us, could help us be kinder and gentler toward other real people. We all walk around with a huge amount of emotions, anxiety, things we can’t express to ourselves or to others.”
But we could say these things to a virtual being.
Lucy is a cute girl who believes there are wolves in the walls of her house. She’s also a companion who looks you in the eye and talks with you, remembering the choices that you’ve made.
“Lucy is a kid, our first being, from a Neil Gaiman story,” Saatchi said. “You will be able to have video chats quite soon, interact with her, learn from her. We haven’t yet seen a connection between a character’s life you can follow on Instagram or YouTube, but also be able to talk to that character, check in with them, talk about what is going on with you, build empathy. For us, this is the first time you can interact with a character and build a set of memories and an emotional relationship with that character.”
Fable Studio is building out that interactive side of Lucy, as well as with other characters as well. Maybe Lucy will show up in Zoom meetings in the future, helping create the illusion that this character is real.
“She sees things in a hopeful and optimistic way,” Saatchi said. “In the conversations we have with Lucy, you get a sense of her as a real, three-dimensional character that is hopefully connective. In this coronavirus period, for us, it is always important for us to wake up in the morning feeling like you are doing something useful. Building a virtual being you can have conversations with, have a video chat with, and communicate with, feels even more important because the loneliness and feeling of isolation — the stuff we think about metaphorically — is real.”
In the story of Lucy, the fact that no one listens to Lucy becomes a metaphor for what is going on in the family. When the wolves do come out of the walls, a crisis happens, and she has to take charge and lead her family to love each other and fight the wolves.
“We want you to feel like you can communicate with Lucy as your imaginary friend,” Saatchi said. “We want you to feel that there is a universe we have created and a logic to why you are communicating with each other that is not transactional. It’s not a virtual assistant relationship. It’s more of a story-driven, playful relationship.”
Do digital humans have to look realistic?
Above: The Unreal Engine 5 will produce outstanding imagery for the PlayStation 5.
Other visions of digital humans tend to push the envelope on realism. Epic Games’ is creating highly realistic demos of digital humans (like with its recent Unreal Engine 5 demo for the PlayStation 5) that will make use of the growing processing power of PCs and next-generation consoles.
But, as noted, Lucy is a cartoon. Fable’s DNA includes veterans of Pixar (like cofounder Jessica Yaffa Shamash) and Dreamworks (Peter Billington), as well as artificial intelligence experts. And both are necessary to move virtual beings forward. Saatchi said it is easier to animate a character like Lucy. It’s still hard to get characters who look super-realistic, like Magic Leap’s Mica, to look real when they’re speaking to you.
With Lucy, Fable is going beyond the visuals to integrate a wide variety of AI aspects, including machine learning, computer vision, synthetic speech, memory, and computer animation. The idea is to have a narrative with Lucy leading the storyline, but also to create a real companion for you.
“We think memory is the thing that everyone should be working on or exploring in virtual beings because it might unlock some pretty profound things about how humans relate to each other, and eventually to a virtual being,” Saatchi said.
The visuals have improved over the years, Saatchi said, but the behavior, or the brain of the artificial character, hasn’t kept pace. Fable Studio spends less time on the visuals, and it spends more time on “memory” and the deepening one-to-one relationship between the human (you) and the character. The character should know who you are, with a memory that goes back in time about choices you have made.
The explosion of virtual beings
Above: Lil Miquela
Since Saatchi’s first Virtual Beings Summit in July 2019, there has been an explosion of these projects that are taking very different forms. Replika is a text messaging bot that has millions of users who believe the bot is like a virtual best friend. It’s like a form of therapy with enormous potential, Saatchi said.
“You have an ongoing digital conversation with your Replika,” Saatchi said.
Investor Cyan Bannister of Founders Fund has bankrolled a lot of these projects. She saw a virtual concert put on by a Japanese virtual character, Hatsune Miku, and was fascinated with how she could draw crowds.
One of her investments is Brud. On Instagram, you can follow the virtual life of Brud’s Lil Miquela, an artificial influencer who has 2.3 million followers on Instagram.
Beyond that, you can have a conversation with Deepak Chopra via a project being created by AI Foundation. The celebrity musician Grimes created a digital avatar of herself. MuseNet is an AI that creates its own music, like a new Mozart composition. Genies lets celebrities cash in on animated clones of themselves. Saatchi thinks there is enormous potential for virtual beings in this space.
Virtual Immortality is resurrecting deceased actor James Dean for a computer-generated imaging (CGI) performance in a new film.
“I just think that’s fantastic,” Saatchi said. “A lot of people obviously think that’s terrible. Artistically, that’s so fascinating to think about a new performance.”
The Wave has virtual concerts where a real performer like John Legend is driving a motion-captured animated character. Netflix will likely move deeper into interactive, as it did with Black Mirror: Bandersnatch, with interactive experiences that are part game and part live action. Facade is an example of an interactive story game that is very conversational.
On the not-so-good side, deep fakes are being used to make real people say things they never said.
“I think this is an interesting moment for us where we can see the beginning of a new movement,” Saatchi said. “Each of these comes with controversy.”
The intersection of virtual beings and games
Above: Motion capture for digital character Lucy of Wolves in the Walls.
Bannister said at the first Virtual Beings Summit that she would love to have a conversation with her grandmother again, but her grandmother passed away. The startup HereAfter is trying to make that kind of conversation happen, across the divide of death. Saatchi noted that his own children won’t have a chance to meet his mother, because she has passed away.
“It will change human psychology, quite a lot,” Saatchi said, if you could have a conversation with someone who died.
These interactions could yield powerful emotional moments associated with virtual beings, and that could benefit games in particular, Saatchi said.
“We’re pretty close to being at the point where we are able to get these things done,” Saatchi said. “Interactive entertainment is getting more cinematic and story driven. There are empathy games and narrative games. That’s going to be very exciting. When you see how much people connect to or love interactive characters like Ellie in The Last of Us, you can see that looking to the virtual beings community will be very important.”
SpiritAI is helping companies create non-player characters who appear to be like real humans, so the hundreds of characters you may come upon in a game will make it feel more immersive, not fake. Speech Graphics is creating facial animation that is AI driven.
Game designers have focused on “emergent gameplay,” where the events in a game aren’t scripted but emerge from what the player does. These types of games are more immersive because the player feels like they’re in a real world, where anything can happen. But Saatchi said much of that is based on environment-based gameplay, like how enemies might spawn in a part of the world where you don’t expect.
But “character-based gameplay” is more of a rarity, and that’s something the virtual beings community is focused on, he said.
“It would be cool to see emergent gameplay where the characters are changing in an unpredictable way,” Saatchi said.
Source: Read Full Article