What are we really doing when we give a task to an AI agent? We're translating that fleeting “intention” in our heads into words.
The agent takes those words, transforms them into something of its own, and produces output. Then we look at it and say “no… that's not what I meant, there's so much more to it…”
But wait — what did we mean?
When we take this question seriously, things get interesting. Because most of the time, we don't fully know what we want ourselves. Do we?
There's a feeling in our heads, a direction, but when put into words something always falls short. If it were a person across from me, they'd understand from my tone, my face, our shared history. They'd fill in the gaps.
With an agent, the only bridge between us is “words”.
Guy Deutscher's Through the Language Glass comes to mind. Language doesn't just carry thought, it shapes it. Different languages carve up the same reality differently, and we mistake the carved shape for reality itself.
The agent speaks a language too. It looks like mine. Fluent, but not its mother tongue. I shape my intention within my own frame; it takes the same words and re-translates them in its own.
Sometimes it aligns with what I meant, sometimes it doesn't. And the difference between the two isn't visible from the outside.
So the real question isn't “why does the agent make mistakes”.
Maybe it's this: how well do we know our own intentions, and how well can we express them?
Working with these systems teaches us something about ourselves, I think.
That we don't think as clearly as we think we do.