The window: a stream that survives a refresh

Read commit 5d117e6 on GitHub · chuyeowego/not-your-slave

Background

Deep background: why streaming chat UIs are harder than they look

Skip if you have built one before.

A chat UI has to solve three problems that pull against each other. It must show text as it arrives, so the page feels alive. It must survive a reload, because people reload. And it must not show the same message twice, which sounds trivial until the first two requirements are in place.

The usual solution is for the client to keep its own copy of the conversation: append each message to an array, persist it, render from it. That works and it is what most chat apps do. It also means two sources of truth - the server's history and the client's array - and every bug in a chat UI you have ever hit is the seam between them.

eve offers a way out. Every event of a session is durably recorded before its step completes, and the stream endpoint takes an absolute cursor:

GET /eve/v1/session/:id/stream                       # follow live
GET /eve/v1/session/:id/stream?startIndex=0          # replay from the beginning
GET /eve/v1/session/:id/stream?startIndex=-1         # start at the tail
GET /eve/v1/session/:id/stream?startIndex=12&includeTailIndex=1   # catch up, then stop

If the server can replay the conversation on demand, the client does not need to store it. That single fact shapes the whole page.

Narrow background: custom channels

A file in agent/channels/ can declare HTTP routes with GET and POST helpers, and each route receives operation surfaces - most importantly from(address), which binds send and friends to a channel-local conversation address, creating a session if that address has none.

Intuition

The interesting design decision is the one that got made twice. Here is the first version, which is what you write if you think of a stream as a request:

The bug it produces is instructive. Open a stream with no cursor and you get the session from the beginning, so the events of the previous turn arrive first. The page happily paints the previous answer into the bubble you just created for the new one - so you ask a question, and watch the answer to your last question appear above your reply that has not been written yet. It looks like a race condition. It is actually a cursor problem.

Concretely, this commit, with a session that already has a two-turn history:

What the page looks like

The code

The channel: 25 lines

export default defineChannel({
  routes: [
    GET("/", async () =>
      new Response(PAGE, { headers: { "content-type": "text/html; charset=utf-8" } })),

    GET("/api/mindlog", async (request) => {
      const limit = Number(new URL(request.url).searchParams.get("limit") ?? 80);
      return Response.json({ entries: await read(Number.isFinite(limit) ? limit : 80) });
    }),

    // Fire a heartbeat by hand. `eve dev` never runs cron, so this is how you
    // watch it think without waiting for production's next tick.
    POST("/api/think", async (_request, { from }) => {
      const session = await from("heartbeat").send(HEARTBEAT, { auth: null });
      return Response.json({ sessionId: session.id });
    }),
  ],
});

In this commit, chat was conspicuously absent: the page talked to eve's own /eve/v1/session API for that, so the channel only supplied what eve did not already have. Three things to notice.

• /api/think sends the exported HEARTBEAT from commit 4. The button is not a simulation of a wake-up, it is the same text cron sends - which is what makes it a usable stand-in during development, where cron never fires.
• This commit's from("heartbeat") was a channel-local address, not a session id. Repeated button presses reused that heartbeat session; cron minted a new one each fire. Same prompt, different continuity — the gap commit 6 had to reckon with. The running tree sends both onto TIMELINE.
• The page is a string in agent/lib/page.ts (a String.raw template) rather than a file read at runtime. It gets bundled with the agent and needs no path resolution in production. The cost is that a literal backtick inside the template ends it - a mistake made and fixed while adding the markdown renderer, resolved by writing backticks as `.

The reader

async function follow(id, startIndex) {
  while (id === sessionId) {
    try {
      const res = await fetch("/eve/v1/session/" + id + "/stream?startIndex=" + startIndex);
      if (!res.ok) return;
      const reader = res.body.getReader();
      …
      for (const line of lines) { handle(JSON.parse(line)); }
    } catch {}
    startIndex = -1;              // reconnect at the tail, not the beginning
    await new Promise((r) => setTimeout(r, 1000));
  }
}

An NDJSON reader with a reconnect loop. In this commit, two details did real work: the loop condition tied the reader's life to the session it was started for, so switching sessions retired the old reader; and the reconnect set startIndex = -1, because a dropped connection had to resume at the tail rather than replay the conversation.

function handle(event) {
  switch (event.type) {
    case "turn.started":      statusEl.textContent = "thinking"; break;
    case "message.appended":  if (!live) live = bubble("it", "it", "");
                              setMessage(live, data.messageSoFar || ""); break;
    case "message.completed": if (data.message) setMessage(live ?? (live = bubble("it","it","")), data.message);
                              live = null; void refreshMindlog(); break;
    case "turn.failed":       bubble("it", "it", "[" + (data.message || "turn failed") + "]"); break;
    case "turn.completed":
    case "session.waiting":   statusEl.textContent = "idle"; void refreshMindlog(); break;
  }
}

Restoring history

async function restore(id) {
  const res = await fetch("/eve/v1/session/" + id + "/stream?startIndex=0&includeTailIndex=1");
  if (res.status === 409) {                  // session is gone
    sessionId = null;
    localStorage.removeItem("nys.session");
    return;
  }
  … for each event:
      if (event.type === "message.received")  bubble("me", "you", data.message);
      else if (event.type === "message.completed" && data.message) bubble("it", "it", data.message);
  void follow(id, -1);                        // then go live
}

In this commit, includeTailIndex=1 asked for a catch-up read that ended rather than following, so this loop terminated on its own and handed over to follow. The whole of "chat history across refreshes" was these fifteen lines. The running tree restores spoken history from the mindlog and attaches the stream at the tail.

Rendering markdown without letting it become markup

The model writes markdown, so the pane renders a small subset of it: paragraphs, bullets, numbered lists, headings, fenced code, and inline bold, italic and code. The safety property is structural rather than a sanitiser:

const node = document.createElement(tag);
node.textContent = bold ?? italic ?? code;   // never innerHTML
into.append(node);
…
if (at < text.length) into.append(text.slice(at));   // plain text as text

Every piece of model output enters the DOM as a text node. There is no path by which a <script> in a reply becomes a script, because nothing is ever parsed as HTML.

The mindlog pane keeps raw text on purpose. It is a log, and a log that reformats its contents is lying slightly: seeing **bold** as characters is the honest view of what was written to the file.

Smaller decisions, briefly

• Theme. Light palette on bare :root, dark redefined under both prefers-color-scheme and [data-theme="dark"], so an unstamped viewer follows the OS and the toggle wins in either direction. The saved choice is stamped in a <head> script before first paint to avoid a flash.
• Resizable panes. A 5px grip between the panes sets a CSS custom property, clamped so neither pane collapses, persisted to localStorage, and arrow-key adjustable for keyboard users. The drag listens on window rather than the grip, so a fast drag that outruns the pointer does not drop the gesture.
• Focus. Any printable key anywhere on the page focuses the composer, so you can just start typing. Modifier combos are left alone so copy still works, as are space and enter on a focused button so the toggle and Wake it still respond to the keyboard.
• Polling. The mindlog pane refetches every three seconds and on every turn boundary.

Quiz

Next: the last commit measures what actually survives a wipe, and closes the gap that measurement exposes.