Chapter 1: It’s Just a JSON Array

Harness Engineering 101, Part I — The Wire. Series index · Next: The Brain

In late 2023 I was building WSO2 MI Copilot, a coding assistant like GitHub Copilot for software integration in WSO2’s Synapse language. It ran on GPT-3.5. There were no agents and no caching, and hardly anyone used tool calls. You sent an array of messages to an HTTP endpoint and got formatted text back, and that was all. Our array held the user’s whole project, sent again with every message.

That is still all that happens, and this whole series is built on that fact. Every agent you have seen, including the one that edits your code and files your pull requests, is a program that builds a JSON array, POSTs it, reads the reply, updates the array, and POSTs it again.

The LLM is the brain, and it can receive text and emit text, nothing else. Whenever it appears to do something in the world, some other program actually did it. That program is the harness: the body around the brain. The only channel between brain and body is the JSON array.

Harness engineering is the work of building and maintaining that array. This chapter is about the array itself.

The call

Remove every SDK and framework, and a call to a frontier model looks like this:

import json, os, urllib.request

def call_llm(messages, system=""):
    body = {
        "model": "claude-sonnet-5",
        "max_tokens": 4096,
        "system": system,
        "messages": messages,
    }
    req = urllib.request.Request(
        "https://api.anthropic.com/v1/messages",
        data=json.dumps(body).encode(),
        headers={
            "content-type": "application/json",
            "x-api-key": os.environ["ANTHROPIC_API_KEY"],
            "anthropic-version": "2023-06-01",
        },
    )
    with urllib.request.urlopen(req) as resp:
        return json.loads(resp.read())

The whole interface is one HTTP POST. A list of dicts goes in and a dict comes out, with no sockets, sessions or handshake.

The messages array is a transcript. Each entry has a role and content:

[
  {"role": "user", "content": "What does HTTP 418 mean?"},
  {"role": "assistant", "content": "It means the server is a teapot. ..."},
  {"role": "user", "content": "Is it ever used seriously?"}
]

You send the array, the model continues it, and the response is the next assistant message.

The most important sentence in this series

The API is stateless. The server remembers nothing between calls.

When you have a “conversation” with a model, there is no conversation stored on the provider’s side. Your program holds the array, appends each new message, and resends the entire history on every turn. The model reads the whole transcript from scratch every time and predicts what comes next. It does not remember writing the earlier messages. It sees a transcript where half the lines are labeled assistant and concludes “apparently I said that.”

sequenceDiagram
    participant H as Harness (your program)
    participant A as API (stateless)
    H->>A: POST [msg1]
    A-->>H: reply1
    Note over H: append reply1, append msg2
    H->>A: POST [msg1, reply1, msg2]
    A-->>H: reply2
    Note over H: append reply2, append msg3
    H->>A: POST [msg1, reply1, msg2, reply2, msg3]
    A-->>H: reply3
    Note over A: remembers nothing,<br/>ever

Once this clicks, a lot of the field gets simpler:

• “Conversation memory” is your program keeping a list.
• “The model forgot something” means the thing fell out of the array.
• “Context management” means deciding what goes in the array.
• “The context window” is the maximum size of the array.
• Cost scales with the array, and you resend it every turn. That becomes chapter 5.

One useful consequence is that sessions are just files. When a coding agent offers --resume, it loads a JSON array from disk and keeps appending. When you type /clear, the implementation is essentially:

messages = []

There is no server-side session to reset. Claude Code stores sessions as JSONL files in a local directory. If you have built a to-do app, you already know how to build session management for an AI agent.

Roles: who is speaking

The array has a small set of speakers. The distinction matters because models are trained to treat each role differently (chapter 2 explains what “trained” means here):

• system: instructions from the developer to the model, such as who it is and what rules it follows. The model is trained to weight this above user text. Anthropic puts it in a top-level system field. OpenAI uses a message with role system, renamed developer in its newer Responses API. All three spellings mean the same thing, a privileged channel for the harness author.
• user: the human’s turn. Later in the series you will see the harness itself use this channel to inject information mid-conversation.
• assistant: the model’s own earlier turns. You wrote none of these, but you store and resend all of them. You can even edit them before resending, and the model can’t tell. That fact becomes a debugging tool in chapter 12 and a safety question in chapter 13.
• Tool results: the outcome of actions, which we will add in chapter 3.

Content is blocks, not strings

Originally, content was a string. It still can be, but in modern APIs it is a list of typed blocks:

{
  "role": "user",
  "content": [
    {"type": "text", "text": "What's wrong with this screenshot?"},
    {"type": "image", "source": {"type": "base64", "media_type": "image/png", "data": "iVBORw0KG..."}}
  ]
}

Text, images (base64 bytes, right in the JSON), PDFs, tool calls and the model’s thinking are all blocks. When models learned to see, the array kept its shape and the blocks got new types. Chapter 2 covers what the brain does with an image block; chapter 3 covers tool blocks. The main thing to remember is that whatever modality or feature ships next year will arrive as a new block type in the same array.

Two dialects, one language

You will meet two wire formats in practice, and the differences are cosmetic. Here is the same exchange in both:

Anthropic Messages API:

POST /v1/messages
{
  "model": "claude-sonnet-5",
  "max_tokens": 1024,
  "system": "You are a terse assistant.",
  "messages": [
    {"role": "user", "content": "Capital of Sri Lanka?"}
  ]
}

OpenAI Chat Completions:

POST /v1/chat/completions
{
  "model": "gpt-5.2",
  "messages": [
    {"role": "system", "content": "You are a terse assistant."},
    {"role": "user", "content": "Capital of Sri Lanka?"}
  ]
}

The differences worth knowing:

OpenAI’s newer Responses API reshuffles this again: input instead of messages, a developer role, and an optional server-side conversation store. That store is the one real departure from statelessness: the provider offers to keep the array for you. Gemini has its own spelling. All of them are an ordered list of role-tagged blocks that is sent whole and continued once.

This is why “which provider” is a shallow decision for a harness. The array is the architecture, and the dialect is a serialization detail. One Code, my provider-neutral rebuild of Claude Code, can treat the provider as a swappable part because everything above the wire format is identical.

The toy harness, v1

Everything in this chapter fits in a program you can read in a minute. This is harness/v1_chat.py, a working chat client with “memory”:

#!/usr/bin/env python3
"""Harness v1: a chat loop. The array IS the conversation."""
import json, os, urllib.request

API_KEY = os.environ["ANTHROPIC_API_KEY"]
MODEL = "claude-sonnet-5"
SYSTEM = "You are a concise assistant."

def call_llm(messages):
    body = {"model": MODEL, "max_tokens": 4096,
            "system": SYSTEM, "messages": messages}
    req = urllib.request.Request(
        "https://api.anthropic.com/v1/messages",
        data=json.dumps(body).encode(),
        headers={"content-type": "application/json",
                 "x-api-key": API_KEY,
                 "anthropic-version": "2023-06-01"})
    with urllib.request.urlopen(req) as resp:
        return json.loads(resp.read())

def main():
    messages = []                                   # the entire "session"
    while True:
        user_text = input("\nyou> ").strip()
        if user_text in ("/quit", ""):
            break
        if user_text == "/clear":
            messages = []                           # "new conversation"
            continue
        messages.append({"role": "user", "content": user_text})
        reply = call_llm(messages)                  # resend EVERYTHING
        text = "".join(b["text"] for b in reply["content"]
                       if b["type"] == "text")
        messages.append({"role": "assistant", "content": reply["content"]})
        print(f"\nassistant> {text}")

if __name__ == "__main__":
    main()

It is about thirty lines of code, and it already has the shape of every harness you will ever build: a loop that appends to an array, sends it whole, and appends the reply. Run it, talk to it, then add print(len(json.dumps(messages))) after each turn and watch the array grow. You are watching your future API bill.

Homework: port call_llm to OpenAI’s Chat Completions format. It is a ten-minute change, and the point is to notice how little changes.

Sidebar: what about streaming? When a chat UI shows the reply appearing word by word, that is the same POST with "stream": true. The server sends the reply in chunks as it generates them. Streaming only changes the presentation. The array and the statelessness are unchanged; the reply just arrives in pieces. This series ignores streaming from here on and loses nothing. When you need it for real (the wire format, assembling the pieces, connections dying mid-reply, and why long requests end up requiring it), Appendix D covers the mechanics.

What the body does

So a harness is “a program that maintains a JSON array.” That sounds like a clerk’s job, but in practice the harness has to decide:

• What enters the array: user text, file contents, tool results, injected instructions (chapters 3, 6, 8).
• What leaves the array: compaction, summarization, forgetting (chapter 6).
• What the array’s structure must preserve: ordering and byte stability, because cost depends on it (chapter 5).
• Which of the brain’s requests to actually execute: permissions, guardrails, sandboxes (chapter 13).
• When to interrupt the brain, and when to wake it (chapters 8 and 9).

None of this is passive plumbing, because the body decides what the brain gets to see, when to interrupt it, and which of its commands to refuse. It behaves less like a set of obedient limbs and more like a body with reflexes.

That framing also explains why this series is not only about coding agents. A coding harness gives the brain hands that edit files and run shells, a robotics harness gives it motors, and a support-desk harness gives it a ticket queue. The bodies differ, but the nervous system (the JSON array and the loop around it) is the same everywhere. This series builds a coding body because it is the easiest to demo in text, but every pattern transfers.

What you now know

• Talking to an LLM is one stateless HTTP POST. You send the whole array every time.
• Memory, sessions and context all live in your program’s array. /clear is messages = [], and resume is a file read.
• Content is a list of typed blocks, and every provider’s API is a dialect of the same structure.

Before we add any features to the harness, we should look at what sits on the other end of that POST, and why how it is trained explains most of its strange behavior.

Next: Chapter 2 — The Brain: A Next-Token Black Box