Cookie Conversation with Jeroen van Loon

IMPAKT Exhibition Don't Be Evil

Jeroen van Loon is an artist who researches and visualises our current digital culture. His work Reinforced Learning shows a compilation of “fail videos”, in which humans have been replaced by robotic humanoids. The work presents a future in which robots and AI have not become “perfect”, but instead have learned from the wrong input: human imperfection.

Reinforced Learning is a very funny approach to a topic that is usually discussed in a more pessimistic or cynical manner. How did the idea for this comedic work come about? And how did you find the videos you wanted to use?

When the pandemic began I started collecting these so-called fail videos on Instagram. First just for my own amusement, I would scroll endlessly at night in bed, looking for some laughs. I often had tears in my eyes from secretly laughing and had to force myself not to laugh out loud which would wake up my partner.

A while later, around 2021, I began thinking about AI and these fail videos, firstly because it would make for a strange and interesting combination: the all-seeing AI and these stupid accidents. I wanted to use a sort of GAN system to generate completely new and imperfect fail videos. That never materialised since the tools and software were not really user-friendly (I cannot code at all) and the overall GAN-aesthetic became utterly boring.
This changed when I saw more and more AI services that could export or capture motion capture data from people in ordinary video’s. That’s when I thought, “oh maybe I can just use the motion capture data and use that for 3D characters”. I started talking to VFX artists that normally work in tv/film industry to figure out how this all could be done when I found a specific AI tool that could do exactly what I needed: extract motion capture data from a person in a video and use that data to superimpose a 3D character in the same video.

The searching for the video’s took me forever, because not every video that I personally found super funny, would work well with a 3D character in it. It seemed that only very specific fail videos worked well. It had to do a lot with the focus of the main character in the video, the camera movement, if the main character was fully in the shot and if there were tiny and small body movements that were not picked up by the AI. It took me a while to figure out all these criteria and limits for selecting fail videos. Apart from all of that I had to think about what kind of fail videos I wanted to include in the final version. Should the robot be failing all the time, should people prank the robot or should the robot prank people? In the end it’s mostly the robot failing in various situations since it creates more empathy to see the robot fail and not people because of the robot.

By presenting humanoid robots in slapstick situations, you criticise the notion that reality can easily be interpreted and predicted by AI systems and robots once enough data has been collected. This can be seen as a counter to utopian as well as dystopian views on artificial intelligence and robotica. How do you currently look at the role of AI and robots in our future?

The main idea was exactly that: showing an alternative future in which robots and AI systems learn from the wrong things, and thus also from our stupid accidents and actions. To me that would also be a more interesting future, because it would perhaps generate more unknown things. In general I’m not opposed to AI services, I think there can be a lot of very useful AI services because they are so good at a single task, much better than any human. For that matter, I could also not have created Reinforced Learning without the VFX AI tool I used.

But from a more fundamental point of view I think that AI is only as good as its training data. Anything that wasn’t in the training data will not be in its output. And there’s a lot of reality that’s often not included in training data, because training data works better if it’s perfected over and over. So an AI or a robot can be very good in what it’s doing, but also very limited compared to our possibilities in reality.

I’m not really sure how to phrase my statement about all of this, but my general feeling is that AI services tend to be boring (artistically speaking) since they are so perfect and have less and less garbage, dirty edges, imperfections, doubts, backtracking, inconsistencies, accidents or just dumb and illogical outputs. Whereas we humans have all of those things and most of the time that’s what makes us interesting. We just try to do our best, and most of the time we fail, but that’s ok.

How does your broader artistic practice relate to the IMPAKT festival we held recently with the theme Our Terms, Our Conditions?

I think both the IMPAKT Festival and I are trying to come up with alternative narratives, tools or tactics for topics within digital culture. I feel this year the IMPAKT Festival was very much focussed on facilitating practical change in the digital sphere, e.g. with the connection between politicians and artists. Which is great because a festival is the perfect venue to bring together people from various fields to discuss the topics they all work on.

The difference between my artistic practice and the festival is that I don’t feel that my works have to be practical or functional. The only thing they have to do is to present an alternative, contrasting or new perspective on something we take for granted, to provide a contrast to the main narrative/tool or tactics. And that’s why I think it works well to have a festival that can both show works of art and connect politicians to these works of art to talk about the practical implications they are suggesting. Because for me often if art is also very useful and functional, it loses some of its power. Just as Reinforced Learning shows: if all data is useful and practical it becomes boring.

In the theme of the festival, this year we created a series of Fortune Cookies with wishes for the future of technology and privacy. Currently, each visitor to the exhibition gets to take one with them. Which fortune cookie wish would you hope to find?
“Experience > Theory.”

Our other Cookie Conversation guests are Renske Leijten, Lotje Beek, Julia Janssen, Dasha Ilina, Caroline Sinders, Guillaume Slizewicz, Roel Heremans, Leon van Oldenborgh and Tomo Kihara.

Website by HOAX Amsterdam