Good afternoon, everyone. Thank you for having me. 

My name is Jed Gjerek. My background is in art, and I’m a PhD candidate in the Department of Political Science and Sociology at the University of Galway, Ireland. Today, I will be talking about Procedural Linguistics – A theory of real-time interactions as a language. 

Very briefly, but important: real-time interactions as a language treat face-to-face communication as a form of language itself. Real-time interactive systems, as a language, treat interactive software architectures as a linguistic model. My research touches on both. 

I want to start with a question. When you walked into a room full of strangers, in the dark, with rain falling all around you — and the rain stopped exactly above your head wherever you moved — what would you think was happening? 

Most people would say: sensors are tracking me, and that’s part of the aesthetic experience. Very few people would say: it’s talking to me. 

But that’s exactly the claim I want to make today. 

If we are back to the installation, it is the Rain Room by Random International Art Collective, first displayed in 2012 at the Barbican Centre here in London. It has been exhibited at MoMA and many other venues ever since. 

In the installation, sensors track your movements, and the system withholds rain wherever you are. You have no instructions; rather, you enter, and you realise you don’t get wet — and then, slowly, you learn that your movement shapes the system’s response. You learn the rules by acting, not by being told. 

And I think that moment — the moment you learn the rules by doing — is a communication. It’s not a metaphor. I think it’s actually a language. Just not the kind we’re used to. 

So let me back up and explain the framework. 

The core idea is simple: real-time interactive systems — algorithms, interfaces, feedback loops, live networks — are typically understood as tools. Things that mediate our lives but sit outside us, neutral, waiting to be used. Imagine a brush which doesn’t care who paints. We assume platforms and systems work the same way. 

I want to challenge that assumption. Because a brush doesn’t train your behaviour through repeated feedback or doesn’t decide what you’re allowed to do next. These systems do. And that makes them closer to a language than a tool. 

I call this framework procedural linguistics. In traditional linguistics studies, meaning is carried by words and symbols, but here the word procedural matters. Think about how you learned to ride a bike. Nobody handed you a manual. You tried, you fell, you adjusted, you tried again, until balance became instinct. That’s procedural knowledge or knowledge built through doing. Procedural linguistics proposes that real-time interactive systems work the same way: they are languages you don’t learn by studying, but by acting inside them. And the fact that you must act in real-time covers both real-time interactive systems as a language and real-time interactions as a language. 

This framework draws on several theoretical traditions – some of them are like the linguistic and pictorial turns, embodied meaning theory, and posthumanism – but they all share one core insight: that meaning is not something stored in our heads and then transmitted through words. Meaning emerges from how we act within the world. 

As a field of research, I use art, particularly interactive art, not only because my background is in art but because artists do something commercial platforms almost never do: they expose the rules. A platform like Instagram or Uber has every intention of hiding its algorithmic logic from you. An artist working with interactive systems often has the opposite motivation — to make the system’s logic visible, even foregrounded, as part of the work itself. So, in interactive artworks, we can see and study the grammar of real-time systems in a relatively pure form, before turning to ask how the same grammar operates in less visible, more consequential settings of digital platforms, gig economy, algorithmic governance, biometric border control, etc. 

So if interactive systems are a language, how do we know that? 

I’d suggest it has three parts.  

First, a syntax or a rule of structure — I’m talking here about the algorithmic rules, limitations, or restrictions that determine what actions are even possible. Like reading from left to right.  

Second, semantics or the meaning of signs — I’m not talking here about the idea behind artwork, so meaning doesn’t come from symbols but from the consequences of your actions within the system.  

Third, there is pragmatic or language in context — and that’s questioning who enters the room and how they act. Also, participants learn to act fluently within that system over time, and their input is a pragmatic choice that alters the artwork’s responses and statements. 

Let’s go back to the Rain Room for a moment, because it makes this more tangible. 

The loop in that installation is simple, and it’s the same loop I see across interactive art generally: there is a system, participant action, system detection, algorithmic processing, system response, participant adjustment. And it goes around and around in a feedback loop. 

What’s important is what happens to you INSIDE in that loop. You’re not interpreting an image. You’re not reading the signs. You are discovering, through repetition, what Ian Bogost calls procedural literacy — you are learning the rules that regulate the system’s behaviour. At that point, something interesting happens. And that’s the point where this theory goes beyond art and art language (which explains why artwork here is just an example or proof). What looked like an artwork or technological interface became, in practice, a communicative structure. Your actions function as statements. The system’s responses function as replies. You are, in a very real sense, having a conversation — just not with words.  

And the same structure we can see everywhere. The same procedural grammar that governs Rain Room governs the gig economy platform that tells a driver where to go next and silently adjusts their pay based on patterns it can sense but never quite reads. It governs the social media feed that doesn’t tell you the rules of what gets seen, but trains you, through repeated feedback, to act in the ways that get rewarded. It governs biometric border control systems, where your body — your walk, your face, your fingerprint — becomes the input to a system whose logic you cannot inspect and whose consequences you cannot escape. 

In every one of these cases, people are not simply being informed to do something. They are being positioned inside a system whose rules they have to learn by doing. That, I think, is the overall impact of my research. If we keep treating these systems as neutral tools — pipes that move information, nothing more — we miss the fact that they can shape our behaviour the way a grammar shapes a sentence by defining, in advance, what can be said or done and what cannot. 

So here is where this leaves us. 

I propose the linguistics of human-machine feedback as an emergent form of language. Not a language that represents the world. Rather, it is a language that produces meaning through specific structures and our possible actions within it. 

This research is still early. What I’ve shared today is a framework, not a finished theory — and much will depend on what the case studies and the artist interviews reveal as the work develops. But I think the direction is a productive one, and I will be happy to answer any questions you may have. 

Thank you very much for your attention.