FutureSafe AI: Algorithmic control & ethics

FutureSafe AI is a speculative design project exploring a future where the state conditions social benefits on genetic analysis. Through a collaborative workshop and future artifacts, we investigated the thin line between technological aid and authoritarian control.

Category:

Speculative Design

Author:

Pavel Danyi

Read:

11 mins

Location:

Prague

Date:

July 21, 2025

The Spark

The project emerged from a personal curiosity about the intersection of technology, reproductive planning, and ethics. It all started with a simple but provocative question:

“What if the state conditioned access to education, healthcare, and other social benefits on whether parents met certain genetic and behavioral criteria?”

This thought experiment became the foundation of a workshop designed to explore how such a system might shape our future. My goal was to create a space where we could reflect on the societal impact of AI while developing artifacts that would bring these abstract ideas to life.

Scenarios and artifacts

The session culminated in the creation of three distinct future scenarios. We explored an optimistic future, where the technology served as a voluntary support tool for parents, and a reflexive scenario, which highlighted the necessity of strict regulatory frameworks. However, the most impactful outcome was the dystopian vision, which illustrated the misuse of technology for state control.

To make this dystopian vision tangible, the team created a specific artifact: a fictional Police Report. This document detailed the case of a woman who refused to undergo the mandatory FutureSafe AI analysis, serving as "evidence" of a future where refusing data collection leads to the systematic restriction of civil rights.

Reflection

This artifact transformed the workshop. It shifted our conversation from abstract theories to concrete realities, demonstrating how thin the line is between a technology that improves lives and one that enforces control. The "Police Report" sparked an intense debate about data privacy and ethics, proving that the issue isn't the technology itself, but its application within specific legal contexts.

Conclusion

Looking back, the workshop successfully fulfilled its purpose. Under the guidance of Roman Sellner, we utilized speculative design to uncover the double edged nature of technological progress. The process emphasized that as we integrate AI into our lives, we must prioritize human rights and equality. FutureSafe AI was not just a speculative exercise, it was a compelling reminder of the responsibility we hold as creators to shape a future that serves people, not just systems.

What is Speculative Design? (From an IKEA Catalog to a Marketplace Ad)

The brief wasn't simply "create an illustration." My task was to apply the method of speculative design. The goal of this method isn't to predict the future, but to materialise possible scenarios – often controversial ones – so we can discuss them before they become reality.

In the course of our collaboration, I generated dozens of different variants and artifacts, from wearables to digital services. For the purpose of this article, however, I will focus on one specific concept that elicited the strongest reactions – the PulseNest.

Roman Sellner and Natálie Káčová provided me with a scenario of the world in 2050, where technology in the care of grieving parents is a common commodity. My task was to create a believable ecosystem for this world. It wasn't enough to just create a product. I had to show its entire life cycle: how it is sold, how it is reported in the news, and what happens to it when no one wants it anymore.

The power of speculative design lies in its banality. If the artifacts looked like props from Black Mirror, the viewer could easily dismiss them as unrealistic. But if they look like something that pops up in your morning news feed or on Facebook, the defense mechanisms crumble, and a sense of chilling unease sets in.

Phase 1: The Commodification of Grief (The Catalog)

The first step was to create a product that promises comfort. I placed the PulseNest artifact within the aesthetic of a department store catalog.

Fig. 1: PulseNest in the aesthetic of a global retail chain. The visual combines a homely atmosphere with a technological description of a product that "plays back the actual heart rhythm and breath of your child." The £990 price tag adds a terrifying reality – grief becomes an item in a shopping cart.

Phase 2: Institutionalization (News 2050)

For the world to be believable, society must react to technology. Therefore, I created a fictional article from the iDNES.cz news server dated 2050.

Fig. 2: Fictional article "PulseNest Enters Czech Hospitals." The text simulates public debate – quoting enthusiastic ministry spokespeople as well as skeptical psychologists warning against "pathological mourning." This artifact demonstrates how such technology would be normalized in the public space.

Phase 3: Waste and Forgetting (Marketplace)

The most chilling part of my work was the visual showing the end of the product's life cycle. What happens to a pillow that simulates the heartbeat of a dead child when the parents no longer want it?

Fig. 3: Ad on Facebook Marketplace. The text "Selling our PulseNest, we no longer need it" for 5,000 CZK is a brutal collision of sacred grief and banal consumerism. It shows the moment when a "tool of love" becomes "used goods" that need to be discarded.

Methodology: battling AI and finding "The Uncanny"

Although the result may look simple, the path to it was long. My main tool was generative artificial intelligence (Perplexity for context, ChatGPT/DALL-E 3 and Gemini for visuals), but it wasn't a "one-click" job.

It was more of a battle with the algorithm. I generated hundreds of variants and variations, testing different styles and compositions. AI tends to generate "pretty" and "happy" people from stock photos. My task was to force it to generate an emotion that is much more complex: a mixture of hope, pain, and shame.

Case study: The evolution of a prompt

What does the search for the right expression look like for a mother using such a product? Here is an example of how I had to iterate the assignment (prompt) to get from kitsch to something that works.

Attempt No. 1 (Failure): Prompt: "Sad mother looking at a computer screen with an avatar of a child, futuristic style, blue light." Result: Too theatrical, too "cinematic." The character was looking directly at the viewer, which felt like a cry for help. It lacked intimacy and believability.

Attempt No. X (Dead End): Prompt: "Product photography, realistic lifestyle shot. A woman sitting at a kitchen table. Looking at a tablet. Warm lighting." Result: Too cozy. It looked like an advertisement for video calls. The uncomfortable ethical contradiction was missing.

Attempt No. 42 (Final Version): Prompt: "Cinematic shot, hyper-realistic. Over-the-shoulder view of a tired woman looking at a tablet screen. The woman is NOT looking at the camera; she is avoiding eye contact, looking down, expression is a mix of comfort and deep shame. Soft, cold morning light. Commercial aesthetic but with a dystopian undertone." Result: Only here did I manage to hit the key emotion – shame. The woman avoids eye contact because she knows what she is doing is socially unacceptable, but she can't help herself. It was this detail, fought for through dozens of attempts, that gave the artifact its believability.

Impact: When Design Causes Terror

Since I was working remotely, I had no direct contact with the workshop participants (healthcare professionals and experts). I often doubted whether mere pixels sent over the internet carried any weight. The true impact of this method only hit me when listening to the podcast Ministerský jednorožec (The Ministerial Unicorn), where psychotherapist Kateřina Hájková Klíčová reflected on the participants' reactions to these very visuals.

The artifacts worked exactly according to the definition of speculative design – they induced an "ethical shock." Participants did not react to the aesthetics, but to the terrifying availability of these technologies. As stated in the podcast:

"I see terror in your eyes. Speculative design can do that. [...] Only when we go through that shock can we then ask: What will we, as humans, do about it?" [^1]

When the healthcare professionals saw the catalog with the PulseNest pillow, their first reaction was rejection: "We don't want this, let's go back to bonfires." And that was exactly the goal. Thanks to these artifacts, they were able to experience a "dry run" of a situation where a parent comes to them requesting such technology. We created a safe simulator for dilemmas that await us.

© Visual Journal ジャーナル

(WDX® — 02)