Events and messages

Two types shape almost every interaction with the SDK: the LilyMessage you send and receive, and the LilyEvent the agent streams while it works.

LilyMessage

A message has a role and content. Content is either a plain string or an array of parts (for vision):

type LilyContentPart =
  | { type: 'text'; text: string }
  | { type: 'image_url'; image_url: { url: string } }

interface LilyMessage {
  role: 'system' | 'user' | 'assistant' | 'tool'
  content: string | LilyContentPart[]
}

Most of the time you send a user message with string content:

await agent.runDetailed([{ role: 'user', content: 'Summarize this article.' }])

For images, use content parts and the poolot-vision model:

const agent = await createLilyAgent({ apiKey, model: 'poolot-vision' })

await agent.runDetailed([
  {
    role: 'user',
    content: [
      { type: 'text', text: 'What is in this image?' },
      { type: 'image_url', image_url: { url: dataUrl } },
    ],
  },
])

You rarely construct assistant or tool messages by hand — the agent produces them, and they show up in result.messages and snapshot().messages.

LilyEvent

While the agent runs, runStreaming (via onEvent) and the onEvent run option emit events. LilyEvent is a discriminated union keyed on type:

typePayloadMeaning
turn_start{ step }A new turn began.
iteration_start{ iteration }The agent loop started an iteration.
iteration_end{ iteration }An iteration finished.
tool_call{ id, name, arguments }The agent is calling a tool.
tool_result{ name, ok, content }A tool returned.
approval_denied{ name }A tool call was denied by an approval rule.
subagent_start{ description }A sub-agent started.
subagent_token{ text }A token from a sub-agent.
subagent_end—A sub-agent finished.
reasoning{ text }Reasoning text (harness only).
reasoning_done{ durationSecs }Reasoning finished.
compaction{ summarizedMessages }History was compacted to save context.
usage{ usage, cumulative }Token usage update.
aborted—The run was aborted.
run_done{ reason, iterations, completedBy }The run ended.
done{ answer }The final answer is ready.

Because it's a discriminated union, TypeScript narrows the payload once you check event.type:

agent.runStreaming(messages, onToken, (event) => {
  switch (event.type) {
    case 'tool_call':
      console.log('calling', event.name, event.arguments)
      break
    case 'tool_result':
      console.log(event.name, event.ok ? 'ok' : 'failed')
      break
    case 'usage':
      updateMeter(event.cumulative)
      break
    case 'done':
      render(event.answer)
      break
  }
})

Answering the agent's questions

The agent can ask the user a question mid-run (an ask_user tool). Register a handler with setInputHandler:

import { setInputHandler } from '@poolot/lily-web'

setInputHandler((question, placeholder, id) => {
  // Return the user's answer as a string, or null to decline.
  return window.prompt(question) // or resolve from your own UI
})

// Clear it when you're done:
setInputHandler(null)

The handler receives the question text, a placeholder, and an id, and returns a string (or null), synchronously or as a Promise. It returns the previous handler so you can restore it.

See also