Skip to main content
The @outputai/llm package is how you call LLMs from your steps and evaluators. It wraps the AI SDK and adds prompt files — version-controlled .prompt files that live alongside your code and define the provider, model, temperature, and prompt template in one place.

Generate Functions

generateText is the primary function for LLM calls. Use the output parameter with Output.* helpers to control the response shape. Use streamText for streaming responses:

Text Output

Generate unstructured text from a prompt file:
steps.ts
result is a convenience alias for response.text.

Streaming

Stream text from a prompt file. Unlike generateText, streamText returns immediately with a stream result — properties like text, usage, and finishReason are promises that resolve when the stream completes.
steps.ts
Note that streamText is not async — it returns a stream result synchronously. Iterate textStream to process chunks as they arrive. You can also await result.text to get the full text in one shot, but that collapses the stream and is functionally identical to generateText. To process chunks with side effects (e.g., writing to stdout):
You can apply stream transforms like smoothStream for more natural output pacing:
Use streaming callbacks for side effects without consuming the stream manually:

Object Output

Generate a structured object matching a Zod schema. This is what you’ll use most in evaluators:
evaluators.ts
output contains the typed object matching your schema.

Image Output

Generate images from a prompt file with generateImage. Image prompt files use plain instructions, not chat role tags like <system> or <user>:
prompts/nascar_race@v1.prompt
Call generateImage from a step:
steps.ts
result is a convenience alias for the first generated image (response.images[0]). The returned image exposes AI SDK image fields such as base64 and mediaType. For image-to-image or edit flows, pass runtime image inputs with images and optionally mask. Output forwards these to the AI SDK prompt object:
Supported image inputs follow the AI SDK shape: Buffer, Uint8Array, ArrayBuffer, raw base64 strings, or { data, mediaType } objects. mask uses the same input shape and requires images.
generateImage does not upload generated images, download remote images, or normalize provider-specific values like size: "auto". Download or upload files in your workflow/client code, pass image bytes to images, and pass concrete provider options through prompt front matter or providerOptions.
Common image options can live in prompt front matter:

Array Output

Generate an array of structured items:

Choice Output

Select one value from a set of options:

Agents

The Agent class wraps AI SDK’s ToolLoopAgent with Output prompt files and the skills system. Use it when you need multi-step tool execution, conversation history, or a reusable agent instance with a fixed configuration. For single-shot LLM calls without tools, generateText is simpler.

Construction

The prompt file is loaded and rendered at construction time. Variables, skills, and tools are fixed at construction. The agent is ready to call generate() or stream() immediately.
steps.ts
Constructor options:

generate()

Run the agent and return when complete:
The result has the same shape as generateText: text, result (alias for text), output, usage, finishReason, toolCalls, etc. Pass additional messages to extend the conversation:

stream()

Stream the agent’s response:
Like streamText, the stream result provides textStream and fullStream iterables, plus promise-based properties (text, usage, finishReason) that resolve on completion.

Structured Output

Use Output.object() with Agent to get typed responses:
steps.ts

Conversation Store

By default, Agent is stateless. Each generate() call starts fresh with only the initial prompt messages. Pass a conversationStore to maintain history across calls:
For custom storage backends, implement the ConversationStore interface:
createMemoryConversationStore() is the built-in in-memory implementation. For production, implement the interface with your database.
stream() does not automatically append messages to the conversation store. If you use streaming with a conversation store, persist messages manually in the onFinish callback.

When to Use Agent vs generateText

Start with generateText. Move to Agent when you need conversation state or a reusable instance with a fixed configuration.

Response Object

generateText returns the full AI SDK response: The cost property is an LLM usage attribute:
Only available, finite usage dimensions are included in usage. For example, reasoning is omitted when the model does not define separate reasoning pricing. Streaming response shape. streamText returns a different result type. Stream iterables (textStream, fullStream) provide real-time chunks, while scalar properties (text, usage, finishReason, etc.) are promises that resolve when the stream completes:

Prompt Files

Instead of hardcoding model config and messages in your code, you write .prompt files that live in your workflow’s prompts/ folder. See the Prompts Guide for the full documentation.
prompts/generate_summary@v1.prompt

Configuration Options

Providers

@outputai/llm ships built-in support for common AI SDK providers. The provider packages are peer dependencies with supported version ranges: Built-in provider instances are initialized lazily. Output creates the provider instance only when a prompt or API call first requests that provider, then reuses it for later calls.

Anthropic

Requires ANTHROPIC_API_KEY environment variable.

OpenAI

Requires OPENAI_API_KEY environment variable.

Azure OpenAI

Requires AZURE_OPENAI_API_KEY, AZURE_OPENAI_ENDPOINT, and AZURE_OPENAI_API_VERSION.

Vertex AI

Requires Google Cloud authentication and configuration.

Amazon Bedrock

Requires AWS credentials (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION) or IAM role-based authentication. Set AWS_SESSION_TOKEN when using temporary credentials (e.g., from aws sts assume-role). For cross-region inference, use the regional inference profile format: us.anthropic.claude-sonnet-4-20250514-v1:0. Always set maxTokens in your Bedrock prompt files. Unlike the direct Anthropic provider (which auto-detects per-model limits), the Bedrock SDK has no client-side defaults and relies on server-side defaults that may be lower than the model’s capacity. When using providerOptions, use the bedrock namespace (not anthropic):

Custom Providers

Use registerProvider when you want prompt files to reference an AI SDK provider that is not built in, or when you need a custom provider instance:
Then use the registered provider name in prompt front matter:
Built-in providers use Output’s default fetch configuration, including longer Undici response timeouts for LLM calls that take time before returning headers or body chunks. Custom registered providers are used exactly as you register them; they do not automatically receive that custom fetch. If your custom provider also needs longer network timeouts, configure its provider instance directly.

Prompt Caching

When a prompt sends the same large prefix on every call — a long system prompt, few-shot examples, a pasted reference document — you can cache that prefix so the provider skips reprocessing it. Cached input is about 90% cheaper and faster to first token. How you enable it depends on the provider.

Anthropic

Anthropic caches only what you explicitly mark. Define a cacheControl set in messageOptions and attach it — with options — to the block that ends your static prefix. Everything up to and including that block is cached and reused on the next call:
prompts/generate_summary@v1.prompt
Only the <user> block — the part that changes each call — is re-billed at full price; the cached <system> prefix is charged at the much cheaper cache-read rate. For the 1-hour cache instead of the default 5 minutes, add ttl: 1h under cacheControl. A block can reference several sets (options="cached fast"), and a set can be reused across blocks.
Attach the set to the last static block, never one containing per-call {{ variables }}. A breakpoint on changing content rewrites the cache on every call and never gets a hit. Order your blocks static-first, dynamic-last.
Each set is a provider-namespaced providerOptions object — the same shape and namespace rules as call-level providerOptions. On Vertex with a Claude model, use the same anthropic namespace.

OpenAI

OpenAI caches automatically — there are no breakpoints to set, so the messageOptions mechanism above isn’t needed. Any prompt of 1024 tokens or longer is cached for you, with no markup. To improve hit rates across calls, set a stable promptCacheKey (and, on GPT-5.1+, extend retention) via providerOptions:
prompts/enrich_company@v1.prompt

Confirming a cache hit

Cache activity appears in the response usage and the cost event: the first call reports cache-creation tokens, and later calls within the TTL report cache-read tokens (cachedInputTokens), already priced at the cheaper rate in response.cost.
Anthropic caches only prefixes above a model-specific minimum — around 1,024 tokens for most Sonnet and Opus models, higher for some. Shorter prefixes are silently not cached, with no error. A request supports at most four cache breakpoints.

Provider Tools

Many providers offer built-in tools like web search. Configure them in YAML front matter:
prompts/research@v1.prompt
This is equivalent to calling vertex.tools.googleSearch({ mode: 'MODE_DYNAMIC', dynamicThreshold: 0.8 }) at the code level, but keeps your prompt self-contained. YAML tools are merged with code-level tools, so you can combine provider tools (from YAML) with custom tools (from code). Code-level tools take precedence if names conflict. For provider-specific tool options, see:

Tool Calling

Use tools with generateText to enable function calling:

AI SDK Pass-Through Options

All generate functions accept additional AI SDK options passed through to the provider: Options set in the prompt file (temperature, maxTokens) can be overridden at call time.

Retries and Network Timeouts

generateText and streamText set maxRetries: 0 by default. In Output workflows, LLM calls usually run inside steps, and steps are Temporal activities. When a provider error is allowed to fail the step, Temporal records the failed activity attempt and retries it according to the workflow’s retry policy. AI SDK retries are still available, but they happen inside one activity attempt. Use them when you want quick provider-level retries before the step fails. Keep the default when you want Temporal to be the single place that controls retries. Pass maxRetries in the function call when you want the AI SDK to retry provider requests:
Built-in providers are initialized with a custom fetch that extends Undici’s headersTimeout and bodyTimeout to 15 minutes. This helps long-running LLM responses where the provider accepts the request but takes longer to return response headers or body chunks, for example reasoning-heavy calls. Active cancellation still works: if you pass abortSignal, or the AI SDK/provider aborts the request, that cancellation wins.

LLM call cost event

Each generateText and streamText call emits a cost:llm:request event after the LLM responds and cost can be computed. You can observe it with the same hooks mechanism as error hooks: register a handler with on('cost:llm:request', handler) from @outputai/core/hooks in a hook file listed under outputai.hookFiles. The handler receives an event envelope whose payload field is the same LLM usage attribute exposed on response.cost. For payload details, see Cost Events.

loadPrompt

Load and render a prompt file without generating — useful for debugging:

API Reference

For complete TypeScript API documentation, see the LLM Module API Reference.