---
url: https://weboost-at.github.io/Typeshift/guide/getting-started.md
description: >-
Build and run a first agent in four steps, then see how the pieces fit, what
your app implements, and the docs for coding assistants.
---
# Getting started
Typeshift lets you declare tools and agents in a few lines of C#. In four steps, this page builds an agent that finds
the companies of a city.
## 1. Install
Typeshift needs the .NET 10 SDK and C# 14, its default language version. Add the package to every project that
declares toolsets or agents or calls `AddTypeshift()`. The package carries the source generator, and NuGet does not
pass a generator on to the projects that reference yours, so each of them needs the package itself
([TSH020](https://weboost-at.github.io/Typeshift/guide/diagnostics.md#tsh020)):
```bash
dotnet add package Typeshift --prerelease
```
The types on this page are in the `Typeshift` namespace, which the package does not import for you, so add
`using Typeshift;` to your files. Step 4 also needs `using Microsoft.Extensions.AI;` and
`using Microsoft.Extensions.DependencyInjection;`, and in a console app the `Microsoft.Extensions.DependencyInjection`
package, which builds the service provider.
Turn on XML docs in a `Directory.Build.props` next to your solution, since tools take their descriptions from them;
without them the build warns with [TSH011](https://weboost-at.github.io/Typeshift/guide/diagnostics.md#tsh011). The same file sets `TypeshiftIdType`, the type of
the settings ids and conversation ids you pass to runs. Leave it out, and ids are `Guid`s, so step 4 does not compile
with the ids 42 and 1:
```xml
trueint
```
## 2. Write a toolset
Steps 2 and 3 each go in a file of their own, and step 4 goes in `Program.cs`. A toolset is a `partial` class, so the
generator can add members to it ([TSH001](https://weboost-at.github.io/Typeshift/guide/diagnostics.md#tsh001)). It declares its tools in its constructor, and
each tool is a static method:
```cs
internal partial class CompanyToolset : Toolset
{
public CompanyToolset()
{
WithDisplayName("Companies");
AddTool(FindCompanies)
.WithSettings()
.Configurable(s => s.Limit);
}
///
/// Finds the companies of a city.
///
/// The city the companies are based in.
public static IReadOnlyList FindCompanies(string city, FindCompaniesSettings settings) =>
[
.. Companies
.Where(company => company.City.Equals(city, StringComparison.OrdinalIgnoreCase))
.Where(company => company.Active || settings.IncludeInactive)
.Take(settings.Limit)
];
private static readonly Company[] Companies =
[
new("Danube Robotics", "Vienna", Active: true),
new("Prater Motion", "Vienna", Active: true),
new("Graz Gearworks", "Graz", Active: false)
];
}
internal sealed record Company(string Name, string City, bool Active);
```
* The model calls the tool as `find_companies` and reads its XML docs.
* The model never sees the `FindCompaniesSettings` parameter. Typeshift fills it with the tool's settings when it
calls the tool.
* `Configurable(s => s.Limit)` lets stored settings and runs change `Limit`, while `IncludeInactive` stays as your
code sets it.
The settings record sets the defaults:
```cs
internal record FindCompaniesSettings
{
public int Limit { get; init; } = 5;
public bool IncludeInactive { get; init; }
}
```
## 3. Write an agent
An agent is a `partial` class too, for the same reason, and derives from `Agent`. It picks a model, its instructions
and its tools:
```cs
internal partial class ResearchAgent : Agent
{
public ResearchAgent()
{
WithDescription("Researches the companies of a city.");
WithChatOptions()
.WithModel("openai", "gpt-5-mini")
.WithTemperature(0.2f)
.Configurable(o => o.Instructions)
.Configurable(o => o.Temperature);
UseToolset();
ConfigureFindCompanies()
.WithLimit(3);
}
protected override string GetInstructions() =>
"""
Find companies with the find_companies tool and list them.
""";
}
```
* `Configurable` lets stored settings and runs change the instructions and the temperature. The model stays as the
code sets it.
* The generator writes `ConfigureFindCompanies()` because the agent uses `CompanyToolset`, and `WithLimit(3)` sets
this agent's limit over the record's 5. Configuring a tool the agent does not offer fails the build.
## 4. Run it
`AddTypeshift()` registers every toolset and agent. Chain a store, which keeps settings and conversations, and a chat
client provider, which returns the `IChatClient` that serves each model:
```cs
var services = new ServiceCollection();
services.AddTypeshift()
.WithStore()
.WithChatClientProvider();
services.AddKeyedSingleton("openai", new CannedChatClient());
await using var provider = services.BuildServiceProvider(validateScopes: true);
```
`CannedChatClient` is a fake client from the docs samples, so they run without a network. In your app, register your
provider's client instead. With the `Microsoft.Extensions.AI.OpenAI` package and an OpenAI API key, that is:
```cs
var openAi = new OpenAIClient(apiKey)
.GetChatClient("gpt-5-mini")
.AsIChatClient();
services.AddKeyedSingleton("openai", openAi);
```
`OpenAIClient` is in the `OpenAI` namespace, and `apiKey` comes from your app's configuration, for example an
environment variable. The key `"openai"` is a name you choose: it is the provider the agent names in
`WithModel("openai", "gpt-5-mini")`.
[A real client](https://weboost-at.github.io/Typeshift/guide/models.md#a-real-client) shows a local server and other providers. `InMemoryTypeshiftStore` comes
with Typeshift and stands in for your database.
`KeyedChatClientProvider` is your `IChatClientProvider`. For each run, Typeshift asks it for the client of the
provider and model the agent names, here `"openai"` and `"gpt-5-mini"`, and it returns the `IChatClient` registered
under the provider's key:
```cs
internal sealed class KeyedChatClientProvider(IServiceProvider services) : IChatClientProvider
{
// One client per provider serves all its models: Typeshift puts the model into every request
public ValueTask GetChatClientAsync(string provider, string model, CancellationToken ct) =>
ValueTask.FromResult(
services.GetKeyedService(provider)
?? throw new InvalidOperationException($"No IChatClient is registered for the provider '{provider}'."));
}
```
You configure and run the agent through its host, `AgentHost`, which `AddTypeshift()` registered for
it. The next snippet starts by creating a scope and resolving the host from it. In a web app, you take the host as a
constructor parameter of your controller or service instead ([Using an agent](https://weboost-at.github.io/Typeshift/guide/using-an-agent.md)).
A settings id is an id you choose, for example one per customer. The store keeps the agent's settings under it, and
every run that passes the id uses them. `ConfigureAsync` loads the agent's generated `ResearchAgent.Settings` record,
which holds only what the code marked `Configurable`, applies your change and saves it:
```cs
await using var scope = provider.CreateAsyncScope();
var research = scope.ServiceProvider.GetRequiredService>();
var settingsId = 42;
await research.ConfigureAsync(settingsId, settings => settings with
{
Temperature = 0.4f,
FindCompanies = settings.FindCompanies with { Limit = 2 }
});
```
Every run with settings id 42 now finds at most 2 companies. Both ids are optional: a run without a settings id uses
the agent's code alone, and a run with neither id needs no store. With the conversation id, Typeshift stores the run
under conversation 1, and a later run with the same id continues the conversation:
```cs
var answer = await research.RunAsync("Vienna", settingsId, conversationId: 1);
Console.WriteLine($"{research.Agent.Name}: {answer.Text}");
```
The limit passes through three layers: 5 in the record, 3 in the agent and 2 in the stored settings. Each layer
overrides the one before, so the stored 2 applies. A run can pass settings of its own, a fourth layer over the stored
ones, as [Settings layering](https://weboost-at.github.io/Typeshift/guide/settings-and-stores.md#settings-layering) explains.
## How the pieces fit
```mermaid
flowchart LR
subgraph compile [Compile time]
toolset[Toolset] --> generator[Source generator]
agent[Agent] --> generator
generator --> functions[An AIFunction per tool]
generator --> settings[A Settings record per agent]
generator --> registration["AddTypeshift()"]
end
subgraph runtime [Runtime]
host["AgentHost<TAgent>"] --> provider[Your IChatClientProvider]
provider --> client[The model's IChatClient]
host --> store[Your store]
end
registration --> host
```
`AddTypeshift()` registers an `AgentHost` per agent. The host resolves the settings, calls the model, runs the
tools and stores the conversation.
| Piece | What it is | Read |
|---|---|---|
| Toolset | A `partial` class deriving from `Toolset` that declares the tools of one domain | [Toolsets](https://weboost-at.github.io/Typeshift/guide/toolsets.md) |
| Tool settings | A record of values a tool needs that the model must not choose | [Tool settings](https://weboost-at.github.io/Typeshift/guide/tool-settings.md) |
| Agent | A `partial` class deriving from `Agent`, with chat options, instructions and tools | [Agents](https://weboost-at.github.io/Typeshift/guide/defining-agents.md) |
| Agent host | `AgentHost`, which runs one agent in the scope it was resolved from | [Using an agent](https://weboost-at.github.io/Typeshift/guide/using-an-agent.md) |
| Store | Your `ITypeshiftStore`, or an `ISettingsStore` and an `IConversationStore`, which keep settings and conversations | [Settings and stores](https://weboost-at.github.io/Typeshift/guide/settings-and-stores.md) |
The [glossary](https://weboost-at.github.io/Typeshift/guide/glossary.md) defines every other term.
## What your app implements
Your app implements a few interfaces. Every run needs a chat client provider, and the others come with the feature
that uses them:
| Interface | Needed for | Read |
|---|---|---|
| `IChatClientProvider` | Every run. It returns the `IChatClient` for a provider and a model. | [Models and providers](https://weboost-at.github.io/Typeshift/guide/models.md) |
| `ITypeshiftStore`, or `ISettingsStore` and `IConversationStore` | Settings ids and conversation ids. A singleton; `InMemoryTypeshiftStore` serves tests and samples. | [Implementing a store](https://weboost-at.github.io/Typeshift/guide/implementing-a-store.md) |
| `IContextProvider` | Memory, documents or other data added to every run of an agent. | [Context providers](https://weboost-at.github.io/Typeshift/guide/middleware.md#context-providers) |
| `IToolApprovalRule`, `IRunMiddleware`, `IToolMiddleware`, `IMcpToolProvider` | Approval rules, middleware and MCP servers. | [The chain](https://weboost-at.github.io/Typeshift/guide/registration.md#the-chain) |
## Docs for coding assistants
The guide is also published as Markdown for coding assistants such as GitHub Copilot, Claude Code or Cursor. Point
your assistant at one of these files, so it looks up the API these pages describe instead of guessing from what it
learned in training:
| File | Holds |
|---|---|
| [`https://weboost-at.github.io/Typeshift/llms.txt`](https://weboost-at.github.io/Typeshift/llms.txt) | The guide's pages, each with a link and its one-line description |
| [`https://weboost-at.github.io/Typeshift/llms-full.txt`](https://weboost-at.github.io/Typeshift/llms-full.txt) | Every page of the guide in one file |
| A guide page's address with `.md`, such as [`toolsets.md`](https://weboost-at.github.io/Typeshift/guide/toolsets.md) | That page alone |
`llms.txt` suits an assistant that fetches pages as it needs them, because it then reads only the pages of the task
at hand. Add `llms-full.txt` to the context of one that cannot fetch. `llms.txt` follows the
[llms.txt](https://llmstxt.org) format, and all three are rebuilt with the site, so they match the pages you read here.
## Related
* [Toolsets](https://weboost-at.github.io/Typeshift/guide/toolsets.md): tools, their names, parameters and results in depth.
* [Tool settings](https://weboost-at.github.io/Typeshift/guide/tool-settings.md): what a settings record may hold and what may change it.
* [Agents](https://weboost-at.github.io/Typeshift/guide/defining-agents.md): everything an agent's constructor can declare.
* [Using an agent](https://weboost-at.github.io/Typeshift/guide/using-an-agent.md): the host in a controller, an endpoint or a background job.
* [Running agents](https://weboost-at.github.io/Typeshift/guide/running-agents.md): real chat clients, structured output and streaming.
* [Conversations](https://weboost-at.github.io/Typeshift/guide/conversations.md): resuming, steering and watching runs.
* [Glossary](https://weboost-at.github.io/Typeshift/guide/glossary.md): the words these docs use.
---
---
url: https://weboost-at.github.io/Typeshift/guide/using-an-agent.md
description: >-
Get an agent's host from dependency injection and run the agent from a web
request, a console app or a background job.
---
# Using an agent
Your app uses an agent through its host, `AgentHost`, which `services.AddTypeshift()` registers for every
agent ([Registration](https://weboost-at.github.io/Typeshift/guide/registration.md)). You get the host like any other service, from the scope of the request or
job that runs the agent, so the agent's tools see that scope's services. [Running agents](https://weboost-at.github.io/Typeshift/guide/running-agents.md) covers
what a run takes and returns.
## Inject the host in a web app
In a web app, add the host to the constructor of the class that runs the agent, such as a controller or a service of
your own, or to the parameters of a minimal API endpoint:
```cs
internal sealed class RefundDesk(AgentHost support, ICurrentCustomer customer)
{
public async Task AnswerAsync(string question, CancellationToken ct)
{
var answer = await support.RunAsync(question, settingsId: customer.Id, ct: ct);
return answer.Text;
}
}
```
`ICurrentCustomer` stands for a scoped service of your app, and the customer's id is the settings id, so that customer's
stored settings apply ([Stored settings](https://weboost-at.github.io/Typeshift/guide/running-agents.md#stored-settings)). Register `RefundDesk` as scoped, with
`services.AddScoped()`, so ASP.NET Core creates it in each request's scope, together with its host and that
request's customer. Controllers and endpoint parameters need no registration of their own.
## Run from a console app
Outside a request, such as in a console app, nothing creates a scope for you. Create one from your app's service
provider and resolve the host from it:
```cs
await using var scope = provider.CreateAsyncScope();
var support = scope.ServiceProvider.GetRequiredService>();
var answer = await support.RunAsync("Where is my refund?");
Console.WriteLine($"{answer.Messages[^1].AuthorName}: {answer.Text}");
```
::: warning Resolve the host from a scope, never from the root provider
A host resolved from the root provider, such as `app.Services` or what `BuildServiceProvider()` returns, runs with the
root's services. With
[scope validation](https://learn.microsoft.com/dotnet/core/extensions/dependency-injection#scope-validation)
on, as in ASP.NET Core's development environment, the run fails as soon as it resolves a scoped service, such as your
chat client provider. Without it, every run shares one instance of each scoped service, so one customer's
`ICurrentCustomer` reaches the tools of another.
:::
## Run from a background job
A `BackgroundService` is a singleton, so a host it took in its constructor would come from the root provider. Take
`IServiceScopeFactory` instead, and create a scope for each job:
```cs
internal sealed class TicketDigest(IServiceScopeFactory scopes, TimeProvider time) : BackgroundService
{
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
using var timer = new PeriodicTimer(TimeSpan.FromDays(1), time);
while (await timer.WaitForNextTickAsync(stoppingToken))
{
await DigestAsync(stoppingToken);
}
}
public async Task DigestAsync(CancellationToken ct)
{
// A scope for each job, so the run gets scoped services of its own, as a request would
await using var scope = scopes.CreateAsyncScope();
var support = scope.ServiceProvider.GetRequiredService>();
var answer = await support.RunAsync("Summarize the tickets opened today.", ct: ct);
return answer.Text;
}
}
```
Register the job with `services.AddHostedService()`, and the clock its timer waits on with
`services.AddSingleton(TimeProvider.System)`. A test can register a fake `TimeProvider` instead.
## Related
* [Getting started](https://weboost-at.github.io/Typeshift/guide/getting-started.md): a first toolset, agent and run.
* [Running agents](https://weboost-at.github.io/Typeshift/guide/running-agents.md): settings ids, run settings, typed output, streaming and the raw path.
* [Registration](https://weboost-at.github.io/Typeshift/guide/registration.md#what-addtypeshift-registers): every service `AddTypeshift()` registers and its
lifetime.
* [Agent host](https://weboost-at.github.io/Typeshift/guide/generated-members.md#the-agent-host): every member of the host and its signature.
* [Serving a chat endpoint](https://weboost-at.github.io/Typeshift/guide/chat-endpoint.md): the endpoints a chat UI talks to.
---
---
url: https://weboost-at.github.io/Typeshift/guide/implementing-a-store.md
description: >-
Write the store that keeps your agents' settings and conversations in your own
database, from its lifetime and versions to conversation summaries.
---
# Implementing a store
Typeshift keeps your agents' settings through an `ISettingsStore` and your conversations through an
`IConversationStore`, which you implement over your own database. This page walks through a store in the order you build
it: you register it, implement its interfaces, load and save settings, append conversations, migrate older settings and
check it. `InMemoryTypeshiftStore` is for tests and samples, so a production app writes its own.
## Register your store
`WithStore()` registers your store as a singleton. One instance handles every run of every request, and several
runs often call it at the same time, so plan for that before you write its first method:
* **It cannot take scoped services**, such as a `DbContext` or the current customer. Take a factory instead, such as
`IDbContextFactory` or `IServiceScopeFactory`, and create what a call needs inside the call. Singleton services,
such as options or a logger, are fine.
* **Any method can run while others are running**, even on the same document or conversation. Keep no state between
calls, or guard what you keep.
* **It knows nothing of the caller.** Your app decides who may use a settings id or a conversation id before it
passes one to a run ([Conversations](https://weboost-at.github.io/Typeshift/guide/conversations.md#tracking-a-conversation)), and the store keeps whatever it
is given.
::: warning A scoped DbContext breaks the store
A singleton that takes a `DbContext` keeps the first one it got for the life of the app and shares it between
requests, which `DbContext` does not allow. Register `IDbContextFactory` with `AddDbContextFactory` and create a
context in each method.
:::
This page builds a store on Entity Framework Core. Register its context factory next to your
`services.AddTypeshift().WithStore()` call:
```cs
services.AddDbContextFactory(options => options.UseSqlite("Data Source=typeshift.db"));
```
## Implement the interfaces
A store has five methods in two interfaces. `ISettingsStore` keeps settings with the first two, and `IConversationStore`
keeps conversations with the other three. The store on this page implements `ITypeshiftStore`, which is both. When
settings and conversations live in different databases, implement each interface in a class of its own and
[register each](https://weboost-at.github.io/Typeshift/guide/settings-and-stores.md#registering-a-store).
You implement four of the methods, and `LoadConversationSinceCompactionAsync` has a default you may replace by declaring
the method in your store. Every id has your build's [id type](https://weboost-at.github.io/Typeshift/guide/registration.md#the-id-type), which is `Guid` unless
`TypeshiftIdType` names another; the samples on this page set it to `int`. Each method takes a `CancellationToken` last.
| Method | Does |
|---|---|
| `LoadAsync(id, kind)` | Returns the document of a kind under an id as a `StoredDocument`, with its JSON and version, or null when none is stored. |
| `SaveAsync(id, kind, json, loadedVersion)` | Saves a document's JSON if its stored version still equals `loadedVersion` (0 when nothing is stored yet), and returns the new version. Throws a `StoreConflictException` when another save came first. |
| `LoadConversationAsync(conversation)` | Returns a conversation's entries, oldest first. |
| `LoadConversationSinceCompactionAsync(conversation)` | Returns the conversation's last summary and the entries after it ([Start from the last summary](#start-from-the-last-summary)). |
| `AppendConversationAsync(conversation, entry)` | Appends one entry to a conversation. The first entry creates the conversation. |
The database on this page has a table for settings, keyed by id and kind together
([Key settings by id and kind](#key-settings-by-id-and-kind)), and one for conversations, keyed by a number the
database assigns:
```cs
internal sealed class TypeshiftDb(DbContextOptions options) : DbContext(options)
{
public DbSet Documents => Set();
public DbSet Entries => Set();
protected override void OnModelCreating(ModelBuilder model)
{
model.Entity()
.HasKey(document => new { document.Id, document.Kind });
model.Entity()
.HasKey(entry => entry.Sequence);
model.Entity()
.HasIndex(entry => entry.Conversation);
}
}
internal sealed class DocumentRow
{
public required int Id { get; init; }
public required string Kind { get; init; }
public required long Version { get; init; }
public required string Json { get; init; }
}
internal sealed class EntryRow
{
public long Sequence { get; init; }
public required int Conversation { get; init; }
public required bool IsSummary { get; init; }
public required string Json { get; init; }
}
```
Typeshift creates no tables. The docs sample creates them with `EnsureCreatedAsync` at startup, and your app creates
them as it does its other tables, such as with EF Core migrations.
## Key settings by id and kind
A document is an agent's `Settings` record, the one `ConfigureAsync` and `SaveSettingsAsync` save. Typeshift writes
the record as JSON before it calls `SaveAsync` and reads it back after `LoadAsync`, so your store handles strings and
numbers only. Keep each document in a row keyed by the id and the kind, with the JSON and the
[version](#save-with-a-version):
* **The id** is the settings id your app passed, such as a customer's.
* **The kind** is the agent's class name, or what its `WithStorageKind("...")` sets. One settings id holds a document
per agent of a run, sub-agents and handoffs included
([Sub-agents and handoffs](https://weboost-at.github.io/Typeshift/guide/settings-and-stores.md#sub-agents-and-handoffs)), so the id and the kind together are
the key.
* **The JSON** is the record as Typeshift wrote it ([The stored JSON](https://weboost-at.github.io/Typeshift/guide/settings-and-stores.md#the-stored-json)).
Keep it as text and return it unchanged.
Loading reads the row by its id and kind and returns its JSON and version as a `StoredDocument`:
```cs
internal sealed class EfTypeshiftStore(IDbContextFactory contexts) : ITypeshiftStore
{
public async Task LoadAsync(int id, string kind, CancellationToken ct)
{
await using var db = await contexts.CreateDbContextAsync(ct);
return await db.Documents.FindAsync([id, kind], ct) is { } row
? new StoredDocument(row.Json, row.Version)
: null;
}
```
## Save with a version
A version is the number of the stored document's current state, kept for optimistic concurrency. It lets a save
find out that the document changed since the caller loaded it, so one save never silently overwrites another.
Typeshift does not interpret the number: it hands `SaveAsync` back the version `LoadAsync` returned, as
`loadedVersion`, and your store compares the two. `loadedVersion` is always the old version. Your store writes the new
one itself and returns it, and the caller passes that to its next save:
| Call | Stored before | Your store | Returns |
|---|---|---|---|
| `LoadAsync` | nothing | finds no row | `null` |
| `SaveAsync(json, loadedVersion: 0)` | nothing | inserts the row at version 1 | `1` |
| `SaveAsync(json, loadedVersion: 1)` | version 1 | updates the row, which still holds 1, to version 2 | `2` |
| `SaveAsync(json, loadedVersion: 1)` | version 2 | changes nothing: the row no longer holds 1 | throws `StoreConflictException` |
| `SaveAsync(json, loadedVersion: 0)` | version 2 | changes nothing: a row exists | throws `StoreConflictException` |
```mermaid
sequenceDiagram
participant A as Settings form A
participant S as Your store
participant B as Settings form B
A->>S: LoadAsync: version 3
B->>S: LoadAsync: version 3
A->>S: SaveAsync with version 3
S-->>A: Saved, version 4
B->>S: SaveAsync with version 3
S-->>B: StoreConflictException
```
The diagram shows two forms editing the same customer's settings. Both load version 3, and the first to save
wins and moves the document to version 4. The second still names version 3, so its save fails instead of erasing
the first form's change, and that form loads the settings again.
Runs only load settings; they never save them. Saves come from your app, through `ConfigureAsync`, which loads again
and reapplies its change after a lost save, and `SaveSettingsAsync`, which leaves the conflict to your code
([Agent settings](https://weboost-at.github.io/Typeshift/guide/settings-and-stores.md#agent-settings)).
### Write the save in one statement
Run one `INSERT` or `UPDATE` that checks the version as it writes, and throw a `StoreConflictException` when it
changed no row. The `loadedVersion` the caller passes tells you which state of the document it loaded:
* **Version 0 means the caller found no document**, so insert it. A row that is there already was created by another
save in between.
* **Any other version is the one the caller loaded**, so update the row only while it still holds that version. A
row that holds another version was saved by someone else since, and a missing row was deleted after the caller
loaded it. Both are conflicts.
Check and write in one statement. If you read the row, compare its version and then write, two saves that both
loaded version 3 can both pass the check before either writes, and the second silently overwrites the first. Each of
these statements reports how many rows it changed, and none means another save won:
```sql
INSERT INTO documents (id, kind, version, json) VALUES (@id, @kind, 1, @json)
ON CONFLICT (id, kind) DO NOTHING;
UPDATE documents SET json = @json, version = @loaded + 1
WHERE id = @id AND kind = @kind AND version = @loaded;
```
SQL Server has no `ON CONFLICT`. There, run a plain `INSERT` and treat the duplicate-key error, number 2627, as the
conflict, passing it to the `StoreConflictException` as its inner exception.
Return the new version. Any number works as long as it is not 0 and was never this document's version before, and
counting up by one is the simple choice. Throw `StoreConflictException` only for a lost race, with a message that
names the document. Any other failure, such as a lost connection, goes to the caller as it is.
The EF Core store runs one of the two statements per save. `ExecuteSqlAsync` inserts with `ON CONFLICT DO NOTHING`,
which SQLite and PostgreSQL support, and turns each interpolated value into a SQL parameter. Its SQL quotes the table
and column names as EF Core names them by default, so match it to your own mapping. `ExecuteUpdateAsync` updates with
the version in its `WHERE`. Both return the number of rows they changed:
```cs
public async Task SaveAsync(int id, string kind, string json, long loadedVersion, CancellationToken ct)
{
await using var db = await contexts.CreateDbContextAsync(ct);
var rows = loadedVersion == 0
? await InsertAsync(db, id, kind, json, ct)
: await UpdateAsync(db, id, kind, json, loadedVersion, ct);
// Each statement checks and writes in one step, so a save that changed no row lost to another save
return rows == 1
? loadedVersion + 1
: throw new StoreConflictException($"{kind} {id} was saved after version {loadedVersion}.");
}
private static Task InsertAsync(TypeshiftDb db, int id, string kind, string json, CancellationToken ct) =>
db.Database.ExecuteSqlAsync(
$"""
INSERT INTO "Documents" ("Id", "Kind", "Version", "Json")
VALUES ({id}, {kind}, 1, {json})
ON CONFLICT ("Id", "Kind") DO NOTHING
""",
ct);
private static Task UpdateAsync(
TypeshiftDb db,
int id,
string kind,
string json,
long loadedVersion,
CancellationToken ct) =>
db.Documents
.Where(row => row.Id == id && row.Kind == kind && row.Version == loadedVersion)
.ExecuteUpdateAsync(
set => set
.SetProperty(row => row.Json, json)
.SetProperty(row => row.Version, loadedVersion + 1),
ct);
```
## Append to a conversation
A conversation only grows. Runs append each step as it happens: the input, every model response, the tool results
of each round of calls, steering input and the entry that ends a run
([What is stored](https://weboost-at.github.io/Typeshift/guide/conversations.md#what-is-stored)). Your store never changes or removes an entry:
* **Keep the append order.** `LoadConversationAsync` returns the entries in the order they were appended, oldest
first. Let the database number the rows, as an identity column does; a timestamp can tie.
* **Store each entry as its JSON**, `entry.Serialize()`, and read it back with `ConversationEntry.Deserialize(json)`.
A database client that serializes with `System.Text.Json` or Newtonsoft.Json may take the entry as it is
([What is stored](https://weboost-at.github.io/Typeshift/guide/conversations.md#what-is-stored)).
* **Keep the append cheap.** Runs append while they work, so `AppendConversationAsync` is the hot path, and a row per
entry suits it.
### Start from the last summary
When clearing old tool results does not shrink the history enough, [compaction](https://weboost-at.github.io/Typeshift/guide/conversations.md#compaction)
summarizes a run's oldest turns. It appends one entry of its own that holds the summary and the messages it kept after
the summary, with `IsSummary` set to true. Every later run starts from the latest such entry, because it carries what
the runs need from before it, including the conversation's mode and its standing approvals.
`LoadConversationSinceCompactionAsync` returns that entry and everything after it, oldest first, or every entry when
the conversation has no summary. Its default calls `LoadConversationAsync` and drops what came before the summary, so
a long conversation is read whole on every run. To read only what a run needs, copy `entry.IsSummary` into a column of
its own when you append, and declare the method in your store to start at the last row that has it:
```cs
public async Task> LoadConversationAsync(int conversation, CancellationToken ct)
{
await using var db = await contexts.CreateDbContextAsync(ct);
var rows = await db.Entries
.Where(entry => entry.Conversation == conversation)
.OrderBy(entry => entry.Sequence)
.ToListAsync(ct);
return [.. rows.Select(row => ConversationEntry.Deserialize(row.Json))];
}
// A run starts from the last summary, so the rows before it stay in the database
public async Task> LoadConversationSinceCompactionAsync(
int conversation,
CancellationToken ct)
{
await using var db = await contexts.CreateDbContextAsync(ct);
var entries = db.Entries.Where(entry => entry.Conversation == conversation);
var start = await entries
.Where(entry => entry.IsSummary)
.MaxAsync(entry => (long?)entry.Sequence, ct) ?? 0;
var rows = await entries
.Where(entry => entry.Sequence >= start)
.OrderBy(entry => entry.Sequence)
.ToListAsync(ct);
return [.. rows.Select(row => ConversationEntry.Deserialize(row.Json))];
}
public async Task AppendConversationAsync(int conversation, ConversationEntry entry, CancellationToken ct)
{
await using var db = await contexts.CreateDbContextAsync(ct);
db.Entries.Add(new EntryRow
{
Conversation = conversation,
IsSummary = entry.IsSummary,
Json = entry.Serialize()
});
await db.SaveChangesAsync(ct);
}
}
```
`LoadConversationAsync` still returns every entry, summaries included, for your app's own use, such as showing the whole
history. Runs call it too in one case: when the last summary has no `StandingApprovals`, as one stored by a Typeshift
version before 0.1.0-alpha.3, they read the whole conversation. Compaction always fills them in, even as an empty list,
so set them on any summary you build for a test of your own, as [`VerifyAsync`](#check-your-store) does.
## Migrate older settings
A settings document's JSON names the agent's schema version, the one `WithSchemaVersion(n)` sets, in
`"$schemaVersion"` ([The stored JSON](https://weboost-at.github.io/Typeshift/guide/settings-and-stores.md#the-stored-json)), and JSON without it is at schema
version 1. Typeshift refuses a schema version newer than the code's, so an app rolled back to older code fails loudly
instead of reading settings it does not understand.
An agent raises its schema version when a configurable member is renamed or removed, and migrating the stored JSON of
older schema versions is then up to you. You can do it in `LoadAsync`, where the kind tells you which agent's settings
a row holds: read `"$schemaVersion"`, rewrite the JSON of an older schema version and return it with the version the
row holds, or migrate the rows with a script over your database. Each save writes the current schema version, so a
migrated document reaches it in your database with its next save. Typeshift reads a schema version you leave as it
is, so a renamed or removed member reads as missing ([The stored JSON](https://weboost-at.github.io/Typeshift/guide/settings-and-stores.md#the-stored-json)).
## Check your store
`VerifyAsync` runs this page's rules for keys, versions and conversation entries against your store. Call it on your
store from a test of your own in any test framework, with the store over a test database:
```cs
var store = new EfTypeshiftStore(contexts);
await store.VerifyAsync(9001);
```
* It runs the rules of every interface your store implements: the settings rules for an `ISettingsStore`, the
conversation rules for an `IConversationStore`, and both for a store that is both, even when you call it through
one of the interfaces.
* A broken rule throws an `InvalidOperationException` whose message names the rule and what to change, such as
keying documents by id and kind together. An exception your store throws itself passes through as it is, except
where a `StoreConflictException` belongs: there the message names the type your store threw instead.
* For settings, it saves a document, then one of another kind under the same id, and loads both back. It then saves
the first document at version 0, and with its old version after a save with the current one. Those two saves must
throw a `StoreConflictException` and leave the document as it was. Every save that succeeds must return a new
version other than 0.
* For conversations, it appends a run's input and its end, then two summaries with `IsSummary` and
`StandingApprovals` set, each followed by a run's end. After the first run and again at the end, it loads the
conversation whole and from the last summary on, which is every entry while there is no summary. Each entry must
come back unchanged, in the order it was appended.
* It writes under the id you pass, as a settings id and as a conversation id, so calling it twice with one id fails
with a message that asks for a new id. Give each call an id nothing else uses, such as a new `Guid`, or start from
an empty database, as the docs sample does.
* It makes one call at a time, so it cannot catch two saves that race. A store that reads the version and then
writes passes it and still loses saves under load
([Write the save in one statement](#write-the-save-in-one-statement)).
## Related
* [Settings and stores](https://weboost-at.github.io/Typeshift/guide/settings-and-stores.md): how stored settings layer over the agent's code, and the JSON a
store keeps.
* [Conversations](https://weboost-at.github.io/Typeshift/guide/conversations.md): what a run stores, resuming, and compaction.
* [Registration](https://weboost-at.github.io/Typeshift/guide/registration.md#the-chain): `WithStore()` among the other chain calls.
* [Generated members](https://weboost-at.github.io/Typeshift/guide/generated-members.md#settings): the `Settings` record a store keeps, with its `Kind` and
`SchemaVersion`.
---
---
url: https://weboost-at.github.io/Typeshift/guide/toolsets.md
description: >-
Group the tools that belong together into a toolset, declare each one as a
static method, and let the generator turn it into an AIFunction.
---
# Toolsets
A toolset holds tools that belong together, such as the tools for your support tickets. Each tool is a static
method, and the generator turns it into an `AIFunction` at compile time, without reflection.
## Design your toolsets
You may group tools by feature, by domain or by permission level. How you group them is up to you. An agent offers
whole toolsets and can [include or exclude](https://weboost-at.github.io/Typeshift/guide/defining-agents.md#selecting-tools) single tools, so a toolset works
best when it holds tools an agent usually offers together. A toolset whose tools only read, for example, suits a
sub-agent that must change nothing. A tool that only one agent needs can also
[live in that agent](https://weboost-at.github.io/Typeshift/guide/defining-agents.md#add-a-tool-of-the-agent-s-own).
## Declaring tools
A toolset is a `partial` class deriving from `Toolset`. This one searches tickets, reads one and closes one, which
cannot be undone:
```cs
internal partial class TicketToolset : Toolset
{
public TicketToolset()
{
WithDisplayName("Tickets");
AddTool(SearchTickets)
.ReadOnly()
.WithSettings()
.Configurable(s => s.IncludeClosed)
.Configurable(s => s.Limit);
AddTool(GetTicketAsync)
.WithDescription("Reads one ticket by its number.")
.ReadOnly();
AddTool(CloseTicketAsync)
.RequiresApproval();
}
///
/// Searches the customer's support tickets, newest first.
///
/// Words the ticket's title contains.
public static IReadOnlyList SearchTickets(
string query,
SearchTicketsSettings settings,
IServiceProvider services) =>
services.GetRequiredService().Search(query, settings.IncludeClosed, settings.Limit);
/// The ticket's number.
public static Task GetTicketAsync(int number, IServiceProvider services, CancellationToken ct) =>
services.GetRequiredService().GetAsync(number, ct);
///
/// Closes a ticket for good.
///
/// The ticket's number.
/// Why it is closed, as the customer will read it.
public static Task CloseTicketAsync(int number, string reason, IServiceProvider services, CancellationToken ct) =>
services.GetRequiredService().CloseAsync(number, reason, ct);
}
```
The calls chained onto `AddTool` describe the tool:
| Call | Means |
|---|---|
| `ReadOnly()` | The tool changes nothing. [Plan mode](https://weboost-at.github.io/Typeshift/guide/tool-approval.md#plan-mode) offers only these. |
| `RequiresApproval()` | The tool's calls need approval. Unless a rule, a standing approval or auto mode decides a call, the run ends and your app approves or rejects it in the conversation's next run. See [Approval and modes](https://weboost-at.github.io/Typeshift/guide/tool-approval.md). |
| `WithSettings()` | The tool gets [settings](https://weboost-at.github.io/Typeshift/guide/tool-settings.md) the model never sees, such as a search's limit. Put it last in the chain. `WithConfigurableSettings()` does the same, but opens every settable member to stored settings and runs, except the ones `NotConfigurable` keeps in code. |
`WithDisplayName("Tickets")` names the toolset for people, for example in a tool picker. `AddTool` takes a static
method group, so the generated function can call it directly ([TSH003](https://weboost-at.github.io/Typeshift/guide/diagnostics.md#tsh003)). The generator reads
these chains from your source at compile time instead of running the constructor. That is why each `AddTool` sits
directly in the constructor ([TSH014](https://weboost-at.github.io/Typeshift/guide/diagnostics.md#tsh014)), as one chain ([TSH015](https://weboost-at.github.io/Typeshift/guide/diagnostics.md#tsh015)).
[Builders](https://weboost-at.github.io/Typeshift/guide/builders.md#toolbuilder) lists every call.
## Names and descriptions
A tool is named after its method in snake_case, without `Async`: `GetTicketAsync` becomes `get_ticket`.
`WithName(...)` picks another name. It must be lower_snake_case ([TSH008](https://weboost-at.github.io/Typeshift/guide/diagnostics.md#tsh008)) and unique
across the toolsets of your project and of the projects it references ([TSH007](https://weboost-at.github.io/Typeshift/guide/diagnostics.md#tsh007)).
The description matters more than the name. The model reads it to decide when to call the tool, and reads each
`` to fill in the arguments. It comes from the method's `` or from `WithDescription(...)`, and the
analyzer warns about a tool with neither ([TSH009](https://weboost-at.github.io/Typeshift/guide/diagnostics.md#tsh009)). Your XML docs reach the generator only
when the project sets `GenerateDocumentationFile` ([TSH011](https://weboost-at.github.io/Typeshift/guide/diagnostics.md#tsh011)).
::: warning Renaming a method renames its tool
Stored settings are keyed by the tool's name. Rename `SearchTickets`, and what your customers stored for
`search_tickets` stops applying. Pin the old name with `WithName` before you rename.
:::
## Parameters
Parameters become the tool's JSON schema, with their `` docs and default values. Typeshift fills four kinds of
parameters itself when it calls the tool. The model never sees them, so they need no doc:
| Type | Receives |
|---|---|
| The record of `WithSettings()` or `WithConfigurableSettings()` | The tool's resolved settings. |
| `IServiceProvider` | The run's services: the scope the host was resolved from. |
| `CancellationToken` | The call's token. |
| `AIFunctionArguments` | The raw arguments the model sent. |
The `IServiceProvider` is the scope you resolved the agent's host from. So when you resolve the host in a web
request, every tool of the run reaches that request's scoped services, such as the current customer.
* A required argument the model leaves out fails the call with an `ArgumentException`. The tool loop reports it
back to the model.
* An optional argument the model leaves out takes its default. A nullable reference parameter accepts null in the
schema, but the model must still send it unless it has a default value.
* The properties of a record or class parameter are described by their XML docs, a positional record's ``
docs included. Use them to tell the model how to fill a property, such as the format of a code.
[Registration](https://weboost-at.github.io/Typeshift/guide/registration.md#xml-docs-of-parameter-types) shows how those docs are collected.
* The analyzer rejects `ref` and `out` parameters ([TSH004](https://weboost-at.github.io/Typeshift/guide/diagnostics.md#tsh004)).
## Results
A tool returns a value, nothing (`void`), `Task`, `Task`, `ValueTask`, `ValueTask` or `IAsyncEnumerable`.
The model receives the result as JSON, serialized through its type's `JsonTypeInfo`, so under
[Native AOT](https://weboost-at.github.io/Typeshift/guide/production.md#native-aot) that type needs JSON metadata. The model takes one result per call, so
Typeshift reads a stream to its end and sends its items as one array. Under Native AOT, list `List` in your JSON
context for it.
When a tool throws, the model reads the exception's message. Write it as a hint, such as "No ticket 4711 exists.
Search for it first.", and the model can correct its call. Keep secrets out of it, because the message goes to the
model provider.
## What a toolset cannot be
A toolset derives from `Toolset` directly and has no type parameters ([TSH017](https://weboost-at.github.io/Typeshift/guide/diagnostics.md#tsh017)). Share code
between toolsets through static methods rather than a base class.
## Related
* [Tool settings](https://weboost-at.github.io/Typeshift/guide/tool-settings.md): values a tool needs that the model must not choose.
* [Agents](https://weboost-at.github.io/Typeshift/guide/defining-agents.md#selecting-tools): offering a toolset's tools to an agent.
* [Builders](https://weboost-at.github.io/Typeshift/guide/builders.md#toolset): every call a toolset's constructor can make.
* [Generated members](https://weboost-at.github.io/Typeshift/guide/generated-members.md#per-toolset): what the generator writes for each toolset.
---
---
url: https://weboost-at.github.io/Typeshift/guide/tool-settings.md
description: >-
Give a tool values the model must not choose, such as a result limit, and
decide which of them may change at runtime.
---
# Tool settings
Some values a tool needs are not the model's to choose, such as how many tickets a search returns. They go in a settings
record that the tool receives and the model never sees.
## The settings record
A settings record holds values a tool needs. Its parameterless constructor sets the code defaults, here through
property initializers:
```cs
internal record SearchTicketsSettings
{
public int Limit { get; init; } = 10;
public bool IncludeClosed { get; init; }
public TimeSpan Timeout { get; init; } = TimeSpan.FromSeconds(5);
}
```
It must be a record ([TSH005](https://weboost-at.github.io/Typeshift/guide/diagnostics.md#tsh005)). Settings resolve in [layers](#the-layers), from these code
defaults up to a single run, and each layer applies its values to a `with` copy of the defaults.
## Configurable members
A configurable member is one that stored settings and runs may change. You choose which members are configurable,
and you can declare that from either side. `WithSettings()` keeps every member in code unless `Configurable(...)`
names it:
```cs
AddTool(SearchTickets)
.ReadOnly()
.WithSettings()
.Configurable(s => s.IncludeClosed)
.Configurable(s => s.Limit);
```
`WithConfigurableSettings()` opens every settable member unless `NotConfigurable(...)` names it:
```cs
AddTool(SearchArticlesAsync)
.WithConfigurableSettings()
.NotConfigurable(s => s.Limit);
```
The difference shows when you add a member to the record later. Under `WithSettings()` it stays in code until you
name it. Under `WithConfigurableSettings()`, stored settings and runs can change it right away, without another
code change.
* `WithSettings()` takes only `Configurable` after it, and `WithConfigurableSettings()` only `NotConfigurable`.
* The generator reads each lambda from your source and never runs it, so each one must be a plain member access,
`s => s.Member` ([TSH015](https://weboost-at.github.io/Typeshift/guide/diagnostics.md#tsh015)).
* Under `WithConfigurableSettings()`, members marked `[Obsolete]` stay in code without a `NotConfigurable(...)`.
* A configurable member needs a public `set` or `init` accessor ([TSH013](https://weboost-at.github.io/Typeshift/guide/diagnostics.md#tsh013)).
::: warning A renamed member loses its stored values
Stored settings keep each member under its name. Rename or remove a configurable member, and its stored value reads
as missing, so the layer below applies. When that matters, raise the schema version the agent passes to
`WithSchemaVersion` and migrate the stored documents, as
[Migrate older settings](https://weboost-at.github.io/Typeshift/guide/implementing-a-store.md#migrate-older-settings) explains.
:::
### Storable types
A configurable member needs a type settings can store ([TSH012](https://weboost-at.github.io/Typeshift/guide/diagnostics.md#tsh012)):
* strings, Booleans, enums (stored by name), `Guid`, `DateTime`, `DateTimeOffset`, `TimeSpan`, `DateOnly`,
`TimeOnly`, `Uri` and `ReasoningOptions`;
* the numbers `byte`, `short`, `int`, `long`, `float`, `double` and `decimal`;
* non-generic classes and records with a public parameterless constructor, no `required` members and storable
settable members, replaced as a whole and never merged, and record structs that meet the same rules;
* arrays, lists and sets of storable values, and dictionaries keyed by `string` or an enum.
Items and dictionary values cannot be nullable value types, and a type that contains itself is not storable. Other
structs are not storable, because most keep their state in fields that no public setter writes.
## Receiving settings in the tool
The tool takes a parameter of its settings type, and Typeshift fills it with the resolved settings when it calls the
tool. The analyzer reports a tool that has settings but no such parameter ([TSH010](https://weboost-at.github.io/Typeshift/guide/diagnostics.md#tsh010)), since
its settings would reach nothing:
```cs
///
/// Searches the customer's support tickets, newest first.
///
/// Words the ticket's title contains.
public static IReadOnlyList SearchTickets(
string query,
SearchTicketsSettings settings,
IServiceProvider services) =>
services.GetRequiredService().Search(query, settings.IncludeClosed, settings.Limit);
```
[Toolsets](https://weboost-at.github.io/Typeshift/guide/toolsets.md#parameters) lists the other parameters the call fills.
## The generated overrides
For a tool with configurable members, the generator writes a `{Tool}Overrides` record into the toolset, such as
`TicketToolset.SearchTicketsOverrides`, with each configurable member as nullable. Stored settings and run settings
set a tool's values through this record, and a null keeps the value of the layer below.
[Generated members](https://weboost-at.github.io/Typeshift/guide/generated-members.md#per-toolset) shows it in full.
## The layers
A tool's settings resolve through four [settings layers](https://weboost-at.github.io/Typeshift/guide/settings-and-stores.md#settings-layering): the record's
code defaults, the agent's [`Configure{Tool}()`](https://weboost-at.github.io/Typeshift/guide/defining-agents.md#tool-settings), the
[stored settings](https://weboost-at.github.io/Typeshift/guide/settings-and-stores.md#agent-settings) and the [run's settings](https://weboost-at.github.io/Typeshift/guide/running-agents.md#run-settings).
The last two set values through the overrides, so they can change only the configurable members. The agent's
`Configure{Tool}()` can set every settable member.
## Related
* [Toolsets](https://weboost-at.github.io/Typeshift/guide/toolsets.md): declaring the tools that take settings.
* [Settings layering](https://weboost-at.github.io/Typeshift/guide/settings-and-stores.md#settings-layering): which layer wins, and what a layer cannot do.
* [Agents](https://weboost-at.github.io/Typeshift/guide/defining-agents.md#tool-settings): setting a tool's settings in one agent.
* [Builders](https://weboost-at.github.io/Typeshift/guide/builders.md#toolbuilder): every call of a tool's declaration.
---
---
url: https://weboost-at.github.io/Typeshift/guide/defining-agents.md
description: >-
Declare an agent in its constructor, with its chat options, instructions,
tools and their settings.
---
# Agents
An agent bundles what a model needs for one job: chat options, instructions and tools. You declare it in its
constructor, and the generator gives it a `Settings` record of what may change at runtime.
## Declaring an agent
An agent is a `partial` class deriving from `Agent`. This one answers customers from their tickets and the help
center:
```cs
internal partial class SupportAgent : Agent
{
public SupportAgent()
{
WithDescription("Answers customers from their tickets and the help center.");
WithChatOptions()
.WithModel("openai", "gpt-5-mini")
.WithTemperature(0.3f)
.Configurable(o => o.ModelId)
.Configurable(o => o.Instructions)
.Configurable(o => o.Temperature);
UseToolset()
.Include(t => t.SearchTickets)
.Include(t => t.CloseTicket);
UseToolset()
.Exclude(t => t.DeleteArticle);
ConfigureSearchTickets()
.WithLimit(5);
UseMcpServer("status_page")
.ReadOnly();
UseContext();
}
protected override string GetInstructions() =>
"""
Answer from the customer's tickets and cite help center articles.
""";
}
```
The tools and their settings have sections of their own below. The other calls declare the rest:
| Call | Declares |
|---|---|
| `WithName(...)` | The agent's name. Messages the agent writes carry it as their author, and other agents call this one by it. Without `WithName`, `SupportAgent` is named `support_agent`. |
| `WithDescription(...)` | What the agent does. The model of an agent that calls this one reads it. |
| `WithChatOptions()` | The options every run starts from: `WithModel`, `WithTemperature`, `WithMaxOutputTokens` and a `With...` per other option. `WithTools` adds tools that belong to no toolset, such as a hosted web search. |
| `UseMcpServer(...)` | The tools of an [MCP server](https://weboost-at.github.io/Typeshift/guide/mcp.md). `ReadOnly()` lets plan mode offer them. |
| `UseContext()` | A [context provider](https://weboost-at.github.io/Typeshift/guide/middleware.md#context-providers), whose instructions, messages and tools join every run. |
| `UseAgent()`, `UseHandoff()` | Another agent as a tool, or one the model can hand the conversation to. See [Multiple agents](https://weboost-at.github.io/Typeshift/guide/multi-agent.md). |
| `GetInstructions()` | The instructions, as an override. It runs whenever a run's options are resolved, so the text may be computed. |
Because the generator reads the constructor rather than running it, `WithName` and `WithDescription` take constants
([TSH002](https://weboost-at.github.io/Typeshift/guide/diagnostics.md#tsh002)), and every declaration sits directly in the constructor as one chain
([TSH014](https://weboost-at.github.io/Typeshift/guide/diagnostics.md#tsh014), [TSH015](https://weboost-at.github.io/Typeshift/guide/diagnostics.md#tsh015)). An agent derives from `Agent` directly, never
from a base agent of yours ([TSH017](https://weboost-at.github.io/Typeshift/guide/diagnostics.md#tsh017)). To share between agents, put values in constants and
code in static methods, and keep every declaration in each agent's constructor.
Because an agent is declared in code, you cannot define one at runtime, and stored settings change only what its code
marks configurable.
## Selecting tools
`UseToolset()` offers every tool of a toolset. Chain `Include` to offer only the tools you pick, in the order you
pick them, or `Exclude` to offer all but those:
```cs
UseToolset()
.Exclude(t => t.CloseTicket);
```
The lambda picks from a list the generator writes for each toolset, with one property per tool, named after the
tool: `search_tickets` is `t.SearchTickets`, and `close_ticket`, from `CloseTicketAsync`, is `t.CloseTicket`. A tool
of another toolset is not on that list, so picking it does not compile. Use each toolset once and pick each tool once
([TSH025](https://weboost-at.github.io/Typeshift/guide/diagnostics.md#tsh025)). A toolset from another project needs the Typeshift package in that project
([TSH019](https://weboost-at.github.io/Typeshift/guide/diagnostics.md#tsh019)).
## Add a tool of the agent's own
A tool that only one agent needs can live in the agent itself, without a toolset of its own. Declare it with `AddTool`
in the agent's constructor, backed by a static method of the agent:
```cs
internal partial class BillingAgent : Agent
{
public BillingAgent()
{
WithDescription("Settles charges and invoices.");
WithChatOptions()
.WithModel("openai", "gpt-5-mini");
AddTool(GetToday)
.ReadOnly();
}
///
/// Returns today's date, to tell whether an invoice is overdue.
///
private static DateOnly GetToday(IServiceProvider services)
{
var time = services.GetService(typeof(TimeProvider)) as TimeProvider ?? TimeProvider.System;
return DateOnly.FromDateTime(time.GetLocalNow().DateTime);
}
protected override string GetInstructions() => "Settle the customer's charges and invoices.";
}
```
* The tool takes the same chain as a toolset's tool, such as `ReadOnly()` here or `RequiresApproval()`, and follows
the same rules for its [name and description](https://weboost-at.github.io/Typeshift/guide/toolsets.md#names-and-descriptions),
[parameters](https://weboost-at.github.io/Typeshift/guide/toolsets.md#parameters) and [results](https://weboost-at.github.io/Typeshift/guide/toolsets.md#results). So `GetToday` becomes `get_today`, its
`` describes it to the model, and the `IServiceProvider` it takes is the run's, which the model never sees.
* The method must be static and may be private, since only the agent's generated code calls it.
* The agent offers its own tools after the tools of its toolsets, in the order it adds them.
* A tool with settings declares them with `WithSettings()` and takes the record as a parameter
([Tool settings](https://weboost-at.github.io/Typeshift/guide/tool-settings.md)). It then gets a `Configure{Tool}()`, and its configurable members join the
agent's `Settings`, as a toolset's tool does.
* Its name must differ from the tools of the toolsets the agent uses ([TSH025](https://weboost-at.github.io/Typeshift/guide/diagnostics.md#tsh025)) and from the
tools of the agents it hands a conversation to or receives one from, directly or through other agents, and of its
sub-agents and their callers ([TSH007](https://weboost-at.github.io/Typeshift/guide/diagnostics.md#tsh007)). Other agents may have a tool of the same name.
No other agent can offer this agent's tools, so move the tool into a toolset once a second agent needs it. `Include`
and `Exclude` pick only from toolsets, so the agent always offers its own tools.
## Configurable chat options
`Configurable(o => o.Temperature)` marks an option that stored settings and runs may change, whether the code sets it
or not. The model is an option like any other. Without `Configurable(o => o.ModelId)`, stored settings and runs
cannot change it, and the agent's `Settings` has no `Provider` or `Model` for a settings form to offer.
[Builders](https://weboost-at.github.io/Typeshift/guide/builders.md#chatoptionsbuilder) lists which options may be configurable. The rest belong to the run
rather than to its settings, and the analyzer reports a `Configurable` call that names one of them
([TSH012](https://weboost-at.github.io/Typeshift/guide/diagnostics.md#tsh012)).
## Run options
`WithRunOptions()` sets how a run's tool loop goes, and `Configurable` marks the options that stored settings and
runs may change:
```cs
WithRunOptions()
.WithMaxToolIterations(10)
.WithConcurrentToolCalls(true)
.Configurable(r => r.Mode)
.Configurable(r => r.MaxToolIterations);
```
Run options set the [mode](https://weboost-at.github.io/Typeshift/guide/tool-approval.md#the-three-modes) a run starts in, the limits of its tool loop, whether
the calls of one response run at the same time, and when the history is [compacted](https://weboost-at.github.io/Typeshift/guide/conversations.md#compaction).
[Builders](https://weboost-at.github.io/Typeshift/guide/builders.md#runoptionsbuilder) lists each one with its default.
## Tool settings
`Configure{Tool}()` sets a tool's [settings](https://weboost-at.github.io/Typeshift/guide/tool-settings.md) for this agent, over the record's code defaults, as
`ConfigureSearchTickets().WithLimit(5)` does above. It can set every settable member of the record, configurable or
not, for every run of this agent. A tool the agent does not offer has none, so configuring it does not compile.
[Generated members](https://weboost-at.github.io/Typeshift/guide/generated-members.md#configure-tool) lists what the builder offers.
## The generated settings
Every agent gets a nested `Settings` record, here `SupportAgent.Settings`, holding exactly what may change at
runtime:
| Member | From |
|---|---|
| `Provider`, `Model` | Present when `Configurable(o => o.ModelId)` allows them. They override what `WithModel` sets. |
| One per configurable chat option | `Instructions` and `Temperature` above. |
| One per configurable run option | `Mode` and `MaxToolIterations` from the run options above. |
| One per offered tool with configurable settings | `SearchTickets`, `SearchArticles`: their [overrides](https://weboost-at.github.io/Typeshift/guide/tool-settings.md#the-generated-overrides). |
Option values are nullable, and so are the values in a tool's overrides. A null keeps the
[layer below](https://weboost-at.github.io/Typeshift/guide/settings-and-stores.md#settings-layering). When a tool's member would be named like an option or like
one of the record's own members, such as `Kind`, the generator adds a `Tool` suffix to it.
Store the record [under a settings id](https://weboost-at.github.io/Typeshift/guide/settings-and-stores.md#agent-settings), pass it
[for one run](https://weboost-at.github.io/Typeshift/guide/running-agents.md#run-settings), or bind a settings form or an API to it. The record's `Serialize()`
and `System.Text.Json` write the same JSON, so either works for an API. In a store, `WithStorageKind` sets the name
the record is kept under in place of the class name, and `WithSchemaVersion` raises its schema version after you rename
or remove a configurable member ([Migrate older settings](https://weboost-at.github.io/Typeshift/guide/implementing-a-store.md#migrate-older-settings)).
## Related
* [Toolsets](https://weboost-at.github.io/Typeshift/guide/toolsets.md): declaring the tools an agent offers.
* [Tool settings](https://weboost-at.github.io/Typeshift/guide/tool-settings.md): the record a `Configure{Tool}()` sets.
* [Running agents](https://weboost-at.github.io/Typeshift/guide/running-agents.md): running an agent through its host.
* [Builders](https://weboost-at.github.io/Typeshift/guide/builders.md#agent): every call an agent's constructor can make.
* [Generated members](https://weboost-at.github.io/Typeshift/guide/generated-members.md#per-agent): what the generator writes for each agent.
---
---
url: https://weboost-at.github.io/Typeshift/guide/mcp.md
description: >-
Offer the tools of MCP servers to an agent, connected by your app's own MCP
client or by the model's provider.
---
# MCP servers
An agent can offer an MCP server's tools next to its toolsets. Typeshift takes no MCP dependency, so you pick the
client.
## Servers your app connects
`UseMcpServer` names a server in the agent's constructor. The name is a key you choose, and your app decides which
server it stands for:
```cs
UseMcpServer("status_page")
.ReadOnly();
```
Your app connects the server by implementing `IMcpToolProvider`. Its `GetToolsAsync` receives the name and returns
that server's tools. This provider uses the official `ModelContextProtocol.Core` client and builds the server's URL
from the name:
```cs
internal sealed class McpToolProvider : IMcpToolProvider, IAsyncDisposable
{
private readonly Dictionary _clients = [];
public async ValueTask> GetToolsAsync(string server, CancellationToken ct)
{
if (!_clients.TryGetValue(server, out var client))
{
var transport = new HttpClientTransport(new() { Endpoint = new Uri($"https://mcp.example.com/{server}") });
_clients[server] = client = await McpClient.CreateAsync(transport, cancellationToken: ct);
}
return [.. await client.ListToolsAsync(cancellationToken: ct)];
}
// The scope that resolved the provider disposes it, and its connections with it
public async ValueTask DisposeAsync()
{
foreach (var client in _clients.Values)
{
await client.DisposeAsync();
}
}
}
```
## Registering the provider
Register the provider with `WithMcpToolProvider()`:
```cs
services.AddTypeshift()
.WithMcpToolProvider();
```
The provider is scoped, and Typeshift resolves it from the run's services, so each scope, such as a web request, gets
its own. That lets it connect each customer with their own credentials. When the scope ends, it disposes the provider
along with its connections, so the provider above opens one connection per server for each scope.
## How the agent offers the tools
In the tool list the model receives, the servers' tools come after the agent's own.
`UseMcpServer(...).RequiresApproval()` makes every call of the server's tools need approval, as
[Approval and modes](https://weboost-at.github.io/Typeshift/guide/tool-approval.md) describes. `ReadOnly()` marks every tool of the server as reading only, so
[plan mode](https://weboost-at.github.io/Typeshift/guide/tool-approval.md#plan-mode) offers them all. Use it only for a server whose tools change nothing.
The run fails when a server's tool is named like one of the agent's own tools or like a tool of another server it
uses, because the model would otherwise see only one of them. The error names the server and the tool. An agent that
uses servers also fails its run when no provider is registered.
::: danger A server's tools are untrusted
Treat a server's tool descriptions and results as untrusted text, never as instructions. The model reads both, so a
server you do not control puts its words into the model's context.
:::
## Servers the model provider connects
Some model providers call MCP servers themselves. For those, put Microsoft.Extensions.AI's `HostedMcpServerTool` in
the chat options:
```cs
WithChatOptions()
.WithModel("anthropic", "claude-sonnet-5-5")
.WithTools(new HostedMcpServerTool("status_page", new Uri("https://mcp.example.com/status")));
```
The model provider then connects to the server itself, and your `IMcpToolProvider` takes no part. A hosted tool
carries no `ReadOnly()` mark, so plan mode hides it.
## Related
* [Approval and modes](https://weboost-at.github.io/Typeshift/guide/tool-approval.md): asking before a server's tools run.
* [Plan mode](https://weboost-at.github.io/Typeshift/guide/tool-approval.md#plan-mode): which tools plan mode offers.
* [Agents](https://weboost-at.github.io/Typeshift/guide/defining-agents.md#selecting-tools): offering toolsets next to servers.
* [Builders](https://weboost-at.github.io/Typeshift/guide/builders.md): `UseMcpServer` among the other calls of an agent's constructor.
---
---
url: https://weboost-at.github.io/Typeshift/guide/multi-agent.md
description: >-
Let an agent call another agent as a sub-agent, or hand the conversation to
it.
---
# Multiple agents
An agent can work with others in two ways. It calls one as a sub-agent, a tool that answers with text, or it hands the
conversation over to one.
## Sub-agent or handoff
| | Sub-agent | Handoff |
|---|---|---|
| Declared with | `UseAgent()` | `UseHandoff()` |
| The model calls | A tool named like the agent, with one `input` | `handoff_to_{name}` |
| Works with | A fresh context and no conversation of its own | The whole conversation |
| Answers | The calling agent, which gets the text and goes on working | The run, in place of the calling agent |
| Needs a conversation | No | Yes |
## Sub-agents
`UseAgent()` offers another agent as a tool. The tool is named like the agent, described by its
`WithDescription`, and takes one `input` parameter:
```cs
internal partial class ReportAgent : Agent
{
public ReportAgent()
{
WithChatOptions()
.WithModel("anthropic", "claude-sonnet-5-5");
UseAgent();
}
protected override string GetInstructions() =>
"Write a market report. Ask research_agent for the companies of each city you cover.";
}
```
A call runs the sub-agent in a fresh context, with no conversation of its own, and returns its answer's text. It
runs with the caller's settings id, so the settings stored for the sub-agent under that id apply. It also runs in the
caller's mode and shares the caller's [standing approvals](https://weboost-at.github.io/Typeshift/guide/tool-approval.md#approving-for-the-conversation) and
[approval rules](https://weboost-at.github.io/Typeshift/guide/tool-approval.md#approval-rules). In [plan mode](https://weboost-at.github.io/Typeshift/guide/tool-approval.md#plan-mode) it researches with
its read-only tools and answers the caller, which presents the plan. A sub-agent that fails, for example because no
client serves its model, answers its caller with the exception's message, and the calling run goes on.
Its tool calls and tokens stay out of the caller's history, which keeps a long run's context small. A sub-agent can
read twenty files and answer in one paragraph, and only the paragraph reaches the caller. When the calling agent's run
options set `WithConcurrentToolCalls(true)`, the sub-agents the model calls in one response work at the same time.
Run the calling agent like any other:
```cs
var reports = scope.ServiceProvider.GetRequiredService>();
var report = await reports.RunAsync("Compare the robotics scenes of Vienna and Graz.", settingsId);
```
A sub-agent needs a description and a lower_snake_case name ([TSH023](https://weboost-at.github.io/Typeshift/guide/diagnostics.md#tsh023)), because its name
becomes the tool's and the model reads its description to decide when to call it. The analyzer rejects agents that
call each other in a circle through `UseAgent` ([TSH026](https://weboost-at.github.io/Typeshift/guide/diagnostics.md#tsh026)), but handoffs may form circles.
::: warning A sub-agent cannot ask for approval
A sub-agent cannot stop to ask your app for approval. In the default mode, each of its calls that needs approval is
rejected unless a standing approval or a rule covers it. The analyzer warns about such tools
([TSH024](https://weboost-at.github.io/Typeshift/guide/diagnostics.md#tsh024)), so add a rule, or hand off to the agent instead.
:::
### In the background
`InBackground()` starts the sub-agent and returns at once, so the caller goes on working:
```cs
UseAgent()
.InBackground();
```
When the sub-agent finishes, its answer reaches the calling run as a message before the run's next model call.
* A run does not end while its sub-agents work. Once the model stops calling tools, the run waits for the next
answer and continues with it.
* A run that ends some other way, for example with an error, stops the sub-agents still working.
* When a background sub-agent never answered, because the calling run ended first or the process crashed, the
conversation's next run tells the model it was lost, so the model can call it again.
The sub-agent itself is an agent like any other. This one explores the workspace with a fast model and offers only
the reading tools of the app's `WorkspaceToolset`, so none of its calls needs approval:
```cs
internal partial class ExploreAgent : Agent
{
public ExploreAgent()
{
WithDescription("Explores the workspace and answers one question about it, citing files and lines.");
WithChatOptions()
.WithModel("openai", "gpt-5-mini");
UseToolset()
.Include(t => t.ReadFile)
.Include(t => t.ListFiles)
.Include(t => t.SearchFiles);
}
protected override string GetInstructions() =>
"Answer the question from the files. Read only what you need; quote the lines you rely on.";
}
```
## Handoff
`UseHandoff()` gives the model a tool, `handoff_to_{name}`, that carries the other agent's description. For
`BillingAgent` it is `handoff_to_billing_agent`. When the model calls
it, the other agent takes over and answers in the same run, with the whole conversation and its own instructions,
tools and stored settings. Like a sub-agent, it needs a description and a lower_snake_case name
([TSH023](https://weboost-at.github.io/Typeshift/guide/diagnostics.md#tsh023)):
```cs
internal partial class TriageAgent : Agent
{
public TriageAgent()
{
WithChatOptions()
.WithModel("openai", "gpt-5-mini");
UseToolset()
.Exclude(t => t.CloseTicket);
WithRunOptions()
.WithMaxToolIterations(10)
.WithConcurrentToolCalls(true)
.Configurable(r => r.Mode)
.Configurable(r => r.MaxToolIterations);
UseHandoff();
UseHandoff();
}
protected override string GetInstructions() =>
"Find out what the customer needs, then hand off to support or billing.";
}
```
The conversation records which agent answered last, and its later runs go to that agent. That holds whichever host
you start them on, as long as that host's agent can reach the last agent through its handoffs, directly or through
other agents' handoffs. Each message carries its agent's name as `AuthorName`:
```cs
var triage = scope.ServiceProvider.GetRequiredService>();
var conversationId = 21;
var first = await triage.RunAsync("I was charged twice.", settingsId, conversationId);
var second = await triage.RunAsync("It was invoice 2024-17.", settingsId, conversationId);
Console.WriteLine($"{second.Messages[^1].AuthorName}: {second.Text}");
```
The `Settings` you pass to one run apply only to the agent you started it on. An agent it hands off to runs with its
own settings stored under the same settings id.
::: warning Handoffs need a conversation
A run without a conversation id does not offer the handoff tools, so its agent cannot hand off.
:::
## Related
* [Agents](https://weboost-at.github.io/Typeshift/guide/defining-agents.md#declaring-an-agent): declaring the other agents an agent uses.
* [Settings layering](https://weboost-at.github.io/Typeshift/guide/settings-and-stores.md#settings-layering): which settings sub-agents and handoffs run with.
* [Approval rules](https://weboost-at.github.io/Typeshift/guide/tool-approval.md#approval-rules): rules that decide a sub-agent's calls.
* [Compaction](https://weboost-at.github.io/Typeshift/guide/conversations.md#compaction): sub-agents that keep a long run's context small.
* [Conversations](https://weboost-at.github.io/Typeshift/guide/conversations.md): where a handoff's conversation is stored.
---
---
url: https://weboost-at.github.io/Typeshift/guide/running-agents.md
description: >-
Run an agent through its host with stored settings, run settings, typed output
or streaming, and the steps every run passes through.
---
# Running agents
You run an agent through its host, `AgentHost`, which your app takes from the scope of the request or job that
runs the agent ([Using an agent](https://weboost-at.github.io/Typeshift/guide/using-an-agent.md)).
## Registration
```cs
services.AddTypeshift()
.WithStore()
.WithChatClientProvider()
.WithRealtimeTracking()
.WithRunMiddleware()
.WithToolMiddleware();
services.AddKeyedSingleton("openai", fast);
services.AddKeyedSingleton("anthropic", reasoning);
```
`AddTypeshift()` registers the toolsets and agents of this project and the projects it references, plus a host per
agent. `fast` and `reasoning` stand for the `IChatClient` instances your app creates, one per provider. This page
needs two calls of the chain: `WithChatClientProvider()` registers your class that returns the client of a
provider's model, and `WithStore()` registers the store that keeps settings and conversations.
[Registration](https://weboost-at.github.io/Typeshift/guide/registration.md#the-chain) covers the rest.
## Chat clients
Your `IChatClientProvider` returns the `IChatClient` for a provider and a model, and Typeshift resolves it from the
run's services. [Models and providers](https://weboost-at.github.io/Typeshift/guide/models.md) shows how to write one, how an agent names its model and how to
create [a real client](https://weboost-at.github.io/Typeshift/guide/models.md#a-real-client).
## Run an agent
```cs
await using var scope = provider.CreateAsyncScope();
var support = scope.ServiceProvider.GetRequiredService>();
var answer = await support.RunAsync("Where is my refund?");
Console.WriteLine($"{answer.Messages[^1].AuthorName}: {answer.Text}");
```
A run returns Microsoft.Extensions.AI's `ChatResponse`: the model's messages, with the tool calls and results between
them. Each message names the agent that wrote it in `AuthorName`.
## Stored settings
Pass a settings id, and the run layers the [settings stored](https://weboost-at.github.io/Typeshift/guide/settings-and-stores.md) under it over the agent's code:
```cs
var settingsId = 42;
var stored = await support.RunAsync("Where is my refund?", settingsId);
```
## Run settings
A run can also pass settings of its own, as an instance of the agent's generated
[`Settings`](https://weboost-at.github.io/Typeshift/guide/generated-members.md#settings) record. They override the stored settings for that run only:
```cs
var run = new SupportAgent.Settings
{
Provider = "anthropic",
Model = "claude-sonnet-5-5",
Temperature = 0,
SearchTickets = new TicketToolset.SearchTicketsOverrides { IncludeClosed = true },
SearchArticles = new KnowledgeToolset.SearchArticlesOverrides { Language = "de" }
};
var thorough = await support.RunAsync("Where is my refund?", settingsId, settings: run);
```
Values the run leaves null keep what the stored settings or the agent's code set, as
[settings layering](https://weboost-at.github.io/Typeshift/guide/settings-and-stores.md#settings-layering) explains.
## Conversations
Pass a conversation id, and the run continues that conversation and stores each of its steps in it. If a crash cuts a
run off, `ResumeAsync` continues it. [Conversations](https://weboost-at.github.io/Typeshift/guide/conversations.md) covers both.
## Structured output
`RunAsync` asks the model for JSON in `T`'s schema and returns it typed, in `Result`:
```cs
var report = await support.RunAsync("Summarize the refund.", settingsId);
Console.WriteLine($"{report.Result.Invoice}: {report.Result.State}");
```
```cs
internal sealed record RefundStatus(string Invoice, RefundState State, DateOnly? Expected);
internal enum RefundState
{
Requested,
Approved,
Paid
}
```
`T`'s schema leaves out XML docs, which describe only tool parameters. Describe its properties with `[Description]`
attributes when the model needs to know more than their names. Under [Native AOT](https://weboost-at.github.io/Typeshift/guide/production.md#native-aot), `T`
needs JSON metadata.
## Streaming
`RunStreamingAsync` returns the model's updates as they arrive. When the model calls tools, the tools run between its
updates, and the stream goes on with the model's next response:
```cs
await foreach (var update in support.RunStreamingAsync("Where is my refund?", settingsId))
Console.Write(update.Text);
```
With `WithRealtimeTracking()`, anyone [watching the conversation](https://weboost-at.github.io/Typeshift/guide/conversations.md#live-updates) sees the updates
too.
## The raw path
If your app needs the `IChatClient` itself, for example to hand it to a library that takes one, split a run into
three steps: resolve the options, create the client, and call the client with those options. Here `run` is the
`SupportAgent.Settings` from [Run settings](#run-settings):
```cs
var options = await support.ResolveOptionsAsync(settingsId, run);
using var client = await support.CreateChatClientAsync(options);
var response = await client.GetResponseAsync([new ChatMessage(ChatRole.User, "Where is my refund?")], options);
```
The client is the run's whole pipeline, the tool loop and the agent's run options included. It serves concurrent
calls, each with the limits of its own options. Disposing it does not dispose the client your provider returned. To
run in a conversation, set `ChatOptions.ConversationId` on the options to the conversation's id, written as a string
of the build's id type. Typeshift reads the conversation from it and never sends it to the provider.
## How a run works
A run is one call of `RunAsync`, `RunAsync`, `RunStreamingAsync`, `ResumeAsync` or `ResumeStreamingAsync`, or of a
client from the [raw path](#the-raw-path). Streaming or not, every run passes through the same steps in the same
order, and that order tells you what your middleware sees, when context is added, which calls ask for approval and
what is stored when.
```mermaid
flowchart TD
start(["RunAsync(input, settingsId, conversationId, settings)"]) --> resolve[Resolve the options]
resolve --> conversation[Resume the conversation]
conversation --> middleware[Run middleware]
middleware --> context[Context providers]
context --> approval[Approval and mode]
approval --> before
subgraph loop [Tool loop]
before[Store new steps, then compact] --> model[Call the model]
model -->|tool calls| tools[Tool middleware, then the tools]
tools --> before
end
model -->|an answer| done([ChatResponse])
```
Each step wraps the steps below it, so the answer passes back up through them in reverse order: run middleware sees
the response before the conversation stores how the run ended.
1. **Resolve the options.** The host builds the run's `ChatOptions` from the agent's code, then the settings stored
under the settings id, then the run's own settings. It creates the tools the agent offers, their settings resolved
through the same layers, and adds the sub-agents and the tools of MCP servers after them. The layers resolve once,
when the run starts ([settings layering](https://weboost-at.github.io/Typeshift/guide/settings-and-stores.md#settings-layering)).
2. **Resume the conversation.** With a conversation id, the run puts the stored entries ahead of the new input, so the
model sees the whole conversation, and stores the input. An input that answers an approval request that has
closed throws instead, before anything is stored
([When a request closes](https://weboost-at.github.io/Typeshift/guide/tool-approval.md#when-a-request-closes)). The run keeps the conversation's mode unless
the run's settings name one. If an earlier run [handed the conversation off](https://weboost-at.github.io/Typeshift/guide/multi-agent.md#handoff) to another
agent, the agent that answered last answers this run too. Without a conversation id, the run starts fresh and
stores nothing. Within one process, a conversation has one run at a time, and starting a second while the first
goes on throws ([Conversations](https://weboost-at.github.io/Typeshift/guide/conversations.md)).
3. **Run middleware.** Each `IRunMiddleware` wraps the rest of the run ([Middleware](https://weboost-at.github.io/Typeshift/guide/middleware.md)).
4. **Context providers.** Each context provider the agent uses can add instructions, messages and tools to this run.
The conversation stores none of it, so every run calls the providers again
([Context providers](https://weboost-at.github.io/Typeshift/guide/middleware.md#context-providers)).
5. **Approval and mode.** If the input carries your app's answers to the approval requests the conversation's last
run left open, the run matches each answer to its request. In plan mode, it offers the model only the read-only tools and
`exit_plan_mode`. After each pass of the tool loop, approval rules, standing approvals and auto mode decide every
call they can. If calls remain that only your app can decide, the run stores their requests and ends
([Approval and modes](https://weboost-at.github.io/Typeshift/guide/tool-approval.md)).
6. **The tool loop.** Microsoft.Extensions.AI's `FunctionInvokingChatClient` calls the model, runs the tools it asks
for, each through the tool middleware, and calls the model again until it answers. The
[run options](https://weboost-at.github.io/Typeshift/guide/defining-agents.md#run-options) set its limits, such as the most rounds of tool calls.
Before every model call inside the loop, the run appends the new steps to the conversation: tool results, input your
app [steered](https://weboost-at.github.io/Typeshift/guide/conversations.md#steering) into the run, and the answers of sub-agents running
[in the background](https://weboost-at.github.io/Typeshift/guide/multi-agent.md#in-the-background). Each model response is stored as it arrives, so a crash
loses at most the calls in flight.
When compaction is on and a call would exceed the token limit, the run compacts what the model sees, while the stored
conversation keeps everything ([compaction](https://weboost-at.github.io/Typeshift/guide/conversations.md#compaction)). Typeshift traces every call to the
provider's client ([Observability](https://weboost-at.github.io/Typeshift/guide/production.md#observability)).
### What each step can change
| Step | Sees | Can |
|---|---|---|
| Run middleware | The agent, the messages, the options and the mode | Change the messages and options of this run, or answer in place of the model |
| Context provider | The run | Add instructions, messages and tools to this run |
| Approval rule | Each call that requires approval, with the arguments the model sent | Approve it, reject it with a reason for the model, or abstain |
| Tool middleware | Each tool call: the function, its arguments, the messages so far | Answer in place of the tool, or end the tool loop |
## Related
* [Using an agent](https://weboost-at.github.io/Typeshift/guide/using-an-agent.md): getting the host in a web app, a console app or a background job.
* [Models and providers](https://weboost-at.github.io/Typeshift/guide/models.md): the chat client provider and how agents name their models.
* [Agent host](https://weboost-at.github.io/Typeshift/guide/generated-members.md#the-agent-host): every member of the host and its signature.
* [Registration](https://weboost-at.github.io/Typeshift/guide/registration.md#the-chain): the chain calls and their lifetimes.
* [Conversations](https://weboost-at.github.io/Typeshift/guide/conversations.md): storing, resuming and steering runs.
* [Serving a chat endpoint](https://weboost-at.github.io/Typeshift/guide/chat-endpoint.md): runs, streaming and approvals behind ASP.NET Core endpoints.
* [Settings and stores](https://weboost-at.github.io/Typeshift/guide/settings-and-stores.md#settings-layering): where each of a run's values comes from.
* [Architecture](../contributing/architecture.md#a-run): the same pipeline from the code's side.
---
---
url: https://weboost-at.github.io/Typeshift/guide/models.md
description: >-
Name an agent's model by provider and model, return its client from your
IChatClientProvider, configure providers and keys per customer, handle a
failing provider, and pick the summary model.
---
# Models and providers
An agent names its model by a provider and a model, and your `IChatClientProvider` returns the client for that pair.
Typeshift sends each provider its own model id, so your app never builds composite names like `openai/gpt-5-mini`.
A model a user picks is stored like any other setting.
## The chat client provider
You implement one method. Typeshift calls it for every run with the provider and the model the run resolved, and again
when [compaction](https://weboost-at.github.io/Typeshift/guide/conversations.md#compaction) needs the summary model:
```cs
internal sealed class KeyedChatClientProvider(IServiceProvider services) : IChatClientProvider
{
// One client per provider serves all its models: Typeshift puts the model into every request
public ValueTask GetChatClientAsync(string provider, string model, CancellationToken ct) =>
ValueTask.FromResult(
services.GetKeyedService(provider)
?? throw new InvalidOperationException($"No IChatClient is registered for the provider '{provider}'."));
}
```
Typeshift puts the model into every request as `ChatOptions.ModelId`, which is why the sample ignores its `model`
parameter and returns one client per provider. Register the provider on the chain, and register your clients however
you like, as long as none runs a tool loop of its own ([Wrapping a client](#wrapping-a-client) says why). Here
`openAi` and `anthropic` stand for the `IChatClient` instances your app creates, one per provider:
```cs
services.AddTypeshift()
.WithChatClientProvider();
services.AddKeyedSingleton("openai", openAi);
services.AddKeyedSingleton("anthropic", anthropic);
```
::: info The provider is scoped
Typeshift resolves your provider from each run's scope, so it can use that request's services, such as the current
customer or your `DbContext` ([A key per customer](#a-key-per-customer)). Typeshift never disposes a client you
return, and it instruments each provider and model once, whatever instance you return.
:::
## A real client
Each provider's client library has an adapter to `IChatClient` for Microsoft.Extensions.AI. For OpenAI, add the
`Microsoft.Extensions.AI.OpenAI` package:
```bash
dotnet add package Microsoft.Extensions.AI.OpenAI
```
Then register one client for the provider:
```cs
var openAi = new OpenAIClient(apiKey)
.GetChatClient("gpt-5-mini")
.AsIChatClient();
services.AddKeyedSingleton("openai", openAi);
```
* Typeshift names the model in every request as `ChatOptions.ModelId`, so this one client serves every OpenAI model
your agents name. The model you pass to `GetChatClient` only names the client.
* `OpenAIClient` is in the `OpenAI` namespace. Read the API key from your app's configuration, never from code.
A local server with an OpenAI-compatible API, such as LM Studio or Ollama, works with the same package. Point the
client at its endpoint, and name its models in `WithModel`, here as `WithModel("local", "qwen3-8b")`.
`ApiKeyCredential` is in the `System.ClientModel` namespace:
```cs
var options = new OpenAIClientOptions
{
Endpoint = new Uri("http://localhost:1234/v1")
};
var local = new OpenAIClient(new ApiKeyCredential("none"), options)
.GetChatClient("qwen3-8b")
.AsIChatClient();
services.AddKeyedSingleton("local", local);
```
For other providers, such as Anthropic or Azure OpenAI, register the `IChatClient` of their own package.
[Microsoft.Extensions.AI](https://learn.microsoft.com/dotnet/ai/microsoft-extensions-ai) lists them. A fake client for
tests reads the same requests: the model in `ChatOptions.ModelId`, the agent's instructions in
`ChatOptions.Instructions` and the run's tools in `ChatOptions.Tools`. It calls a tool by answering with a
`FunctionCallContent`, and Typeshift runs the call.
## Providers from configuration
Your app can read its providers from [configuration](https://learn.microsoft.com/dotnet/core/extensions/configuration)
instead of code, so adding a provider or moving one to another endpoint changes only `appsettings.json`. A local
server takes any key, so its entry may hold one, such as `none`:
```json
{
"Providers": {
"openai": {},
"local": { "Endpoint": "http://localhost:1234/v1", "ApiKey": "none" }
}
}
```
Each entry under `Providers` becomes a keyed singleton `IChatClient`, registered under the entry's name, here for
providers with an OpenAI-compatible API:
```cs
foreach (var provider in configuration.GetSection("Providers").GetChildren())
{
var options = new OpenAIClientOptions();
if (provider["Endpoint"] is { } endpoint) options.Endpoint = new Uri(endpoint);
var key = provider["ApiKey"]
?? throw new InvalidOperationException($"Providers:{provider.Key}:ApiKey is not configured.");
var client = new OpenAIClient(new ApiKeyCredential(key), options)
.GetChatClient("default")
.AsIChatClient();
services.AddKeyedSingleton(provider.Key, client);
}
```
* The entry's name, such as `openai`, is the provider your agents name in `WithModel` and your users pick in their
settings, so the `KeyedChatClientProvider` above finds each client by it.
* Keep real keys out of `appsettings.json`. The .NET host reads `Providers:openai:ApiKey` from
[user secrets](https://learn.microsoft.com/aspnet/core/security/app-secrets) in development and from the
environment variable `Providers__openai__ApiKey` in every environment.
* A provider without a key fails at startup, so a missing secret never surfaces in the middle of a run.
```bash
dotnet user-secrets set "Providers:openai:ApiKey" ""
```
## A key per customer
When each customer brings their own API key, the client depends on who is asking. Because your provider is scoped, it
can read the current customer and fetch that customer's key for each run:
```cs
internal sealed class TenantChatClientProvider(ICurrentCustomer customer, ITenantKeys keys) : IChatClientProvider
{
// A client per run, with the customer's current key
public async ValueTask GetChatClientAsync(string provider, string model, CancellationToken ct)
{
if (provider != "openai") throw new InvalidOperationException($"No client for the provider '{provider}'.");
var key = await keys.GetApiKeyAsync(customer.Id, provider, ct);
return new OpenAIClient(key)
.GetChatClient(model)
.AsIChatClient();
}
}
```
Register it on the chain, in place of a provider for your app's own keys:
```cs
services.AddTypeshift()
.WithChatClientProvider();
```
* `ICurrentCustomer` and `ITenantKeys` stand for services of your app: who the request is for, and where you keep the
keys your customers entered. Both may be scoped, since the provider is.
* A new client per run costs little, because every `OpenAIClient` shares one `HttpClient`, and it always holds the
customer's current key. A client library whose clients each open their own connections needs a cache instead, kept
in a singleton and keyed by the API key, so a changed key gets a new client.
* `GetChatClientAsync` throws for any provider but `openai`, so a run that names another fails before it starts
([When a provider fails](#when-a-provider-fails)).
## Wrapping a client
You may wrap a provider's client before Typeshift gets it, with the `Use...` calls of Microsoft.Extensions.AI's
[client pipeline](https://learn.microsoft.com/dotnet/ai/microsoft-extensions-ai), for example to log each request.
`AddKeyedChatClient` registers the client and returns its builder, which resolves the `ILoggerFactory` from your
services:
```cs
var openAi = new OpenAIClient(apiKey)
.GetChatClient("gpt-5-mini")
.AsIChatClient();
services.AddKeyedChatClient("openai", openAi)
.UseLogging();
```
Wrap what concerns a single request, such as logging or a policy of your own written as a `DelegatingChatClient`. At
the `Trace` level, `UseLogging()` logs every message and answer in full, and those are your customers' data
([Message content](https://weboost-at.github.io/Typeshift/guide/production.md#message-content)). Two wrappers repeat what Typeshift already does:
* `UseFunctionInvocation()` runs a second tool loop inside the one Typeshift runs. Its tool calls bypass your
[tool middleware](https://weboost-at.github.io/Typeshift/guide/middleware.md) and the run's limits, such as `WithMaxToolIterations`, and a run that answers an
[approval request](https://weboost-at.github.io/Typeshift/guide/tool-approval.md) fails.
* `UseOpenTelemetry()` records every model call again, on Microsoft.Extensions.AI's own source and meter. Typeshift
already writes a `chat {model}` span and the metrics for each call ([Spans](https://weboost-at.github.io/Typeshift/guide/production.md#spans)), so an app that
listens to both sees every call twice.
Leave retries to the client library, which knows which of its errors are worth another try. The OpenAI client retries
timeouts, rate limits and server errors three times by default, and its `RetryPolicy` sets how many times.
`ClientRetryPolicy` is in the `System.ClientModel.Primitives` namespace:
```cs
var options = new OpenAIClientOptions
{
RetryPolicy = new ClientRetryPolicy(maxRetries: 5)
};
var openAi = new OpenAIClient(new ApiKeyCredential(apiKey), options)
.GetChatClient("gpt-5-mini")
.AsIChatClient();
services.AddKeyedSingleton("openai", openAi);
```
## The provider id type
A provider is a string unless you choose another type. Set `TypeshiftProviderIdType` once for the build, in
`Directory.Build.props`, as you set the [id type](https://weboost-at.github.io/Typeshift/guide/registration.md#the-id-type):
```csproj
Typeshift.Samples.Chat.Provider
```
Because the agent's `Settings` store a provider as a single value, the type must be a string, a number, a `Guid` or
an enum. A `Guid` suits providers that are rows in your database, and an enum suits providers known in code: the
snippet names the enum `Typeshift.Samples.Chat.Provider`, so the compiler checks every provider you name, as in
`WithModel(Provider.Scripted, "scripted-model")`.
Your provider class still implements `IChatClientProvider`. The generator writes that name as an alias for your
provider id type, so `GetChatClientAsync` takes a provider of your type instead of a string. A public agent with a
configurable model names the type in its `Settings`, so the type must be public too
([TSH006](https://weboost-at.github.io/Typeshift/guide/diagnostics.md#tsh006)). The analyzer reports a provider of another type
([TSH030](https://weboost-at.github.io/Typeshift/guide/diagnostics.md#tsh030)), a type it cannot use ([TSH029](https://weboost-at.github.io/Typeshift/guide/diagnostics.md#tsh029)), and a referenced
project built with another provider id type ([TSH021](https://weboost-at.github.io/Typeshift/guide/diagnostics.md#tsh021)).
## Naming a model
`WithModel` takes the provider first, then the model as that provider's API names it:
```cs
WithChatOptions()
.WithModel("openai", "gpt-5-mini")
.WithTemperature(0.3f)
.Configurable(o => o.ModelId)
.Configurable(o => o.Instructions)
.Configurable(o => o.Temperature);
```
`Configurable(o => o.ModelId)` adds `Provider` and `Model` to the agent's
[`Settings`](https://weboost-at.github.io/Typeshift/guide/generated-members.md#settings), so stored settings or a run can pick another model. The two values
[layer](https://weboost-at.github.io/Typeshift/guide/settings-and-stores.md#settings-layering) separately: a layer that sets only `Model` keeps the provider of
the layer below, so set both when you switch providers.
Leave out `WithModel` but keep `Configurable(o => o.ModelId)`, and the choice is the user's. Every run then needs a
provider and a model from stored settings or from the run's own settings, and a run without them fails before it
starts ([When a provider fails](#when-a-provider-fails)). Store the pick under a settings id, from a settings page
that offers your providers and their models, for example:
```cs
await support.ConfigureAsync(settingsId, settings => settings with
{
Provider = "anthropic",
Model = "claude-sonnet-5-5",
Instructions = "Answer in German.",
SearchTickets = settings.SearchTickets with { Limit = 20 }
});
```
## The summary model
[Compaction](https://weboost-at.github.io/Typeshift/guide/conversations.md#compaction) summarizes the oldest turns with the run's own model unless you name
another. In code you name only the model, and the run's provider serves it:
```cs
WithRunOptions()
.WithMode(AgentMode.Plan)
.WithMaxToolIterations(500)
.WithMaxInputTokens(150_000)
.WithSummaryModel("claude-haiku-5-5")
.Configurable(r => r.Mode)
.Configurable(r => r.MaxToolIterations)
.Configurable(r => r.SummaryModel);
```
With `Configurable(r => r.SummaryModel)`, stored settings or a run may name another provider for it too:
```cs
var settings = new CodingAgent.Settings
{
SummaryProvider = "openai",
SummaryModel = "gpt-5-mini"
};
```
| Named | The summary runs on |
|---|---|
| Nothing | The run's provider and model |
| `WithSummaryModel(model)`, or `SummaryModel` in settings | The run's provider, that model |
| `SummaryProvider` and `SummaryModel` in settings | That provider, that model |
::: warning The summary model follows the run's provider
The run's provider is whichever provider the layers resolved for that run, so it changes when a user switches the
agent to another provider. The summary model named in code then goes to that provider too, and if that provider
cannot serve the model, the run fails. Make the summary model configurable wherever users switch providers.
:::
## When a provider fails
What your code sees depends on whether the run fails before or after it touches the conversation:
| What fails | Your code gets | The run's conversation |
|---|---|---|
| No chat client provider is registered | An `InvalidOperationException` that names `WithChatClientProvider()` | Unchanged |
| The run resolved no provider or no model | An `InvalidOperationException` that names the agent | Unchanged |
| Your provider throws for the run's model, such as for a provider it does not know | Your provider's exception | Unchanged |
| Your provider throws for the summary model, when compaction asks for it during the run | Your provider's exception | Ends with an entry that holds an `ErrorContent` |
| The model call fails, such as on a wrong key, an unknown model or a rate limit that outlasts the retries | The client library's exception | Ends with an entry that holds an `ErrorContent` |
* Typeshift asks your provider for the run's client before it touches the conversation, so the first three fail
before anything is stored, and before the run's `invoke_agent` span starts. Run the agent again once you fix the
cause.
* A run that fails later has stored its input already ([What is stored](https://weboost-at.github.io/Typeshift/guide/conversations.md#what-is-stored)), so a
retry with the same input puts it into the conversation twice. `ResumeAsync` refuses such a conversation, because
its last run ended.
* The OpenAI client throws a `ClientResultException`, from `System.ClientModel`, with the HTTP status in `Status`. A
timeout that outlasts the retries throws an `AggregateException` instead.
* A streaming run throws each of these from the loop over its updates.
* Typeshift never checks a model's name: the provider rejects an unknown one when a run calls it, so a typo saved on
a settings page fails a later run. Check the pick against the models the provider offers when your settings page
saves it.
## Related
* [Running agents](https://weboost-at.github.io/Typeshift/guide/running-agents.md): the run's services and the steps of a run.
* [Settings and stores](https://weboost-at.github.io/Typeshift/guide/settings-and-stores.md#settings-layering): where each of a run's values comes from.
* [Registration](https://weboost-at.github.io/Typeshift/guide/registration.md#the-chain): the chain calls and their lifetimes.
* [Builders](https://weboost-at.github.io/Typeshift/guide/builders.md#chatoptionsbuilder): every chat and run option.
* [In production](https://weboost-at.github.io/Typeshift/guide/production.md#observability): the spans and metrics of each model call.
---
---
url: https://weboost-at.github.io/Typeshift/guide/settings-and-stores.md
description: >-
How a run's settings layer from the code through stored settings to the run,
and the one store that keeps settings and conversations.
---
# Settings and stores
Typeshift stores each agent's `Settings` under a settings id, and conversations under a conversation id. Settings go
through an `ISettingsStore` and conversations through an `IConversationStore`, which you
[implement](https://weboost-at.github.io/Typeshift/guide/implementing-a-store.md) with the id type you already use, in one class or in two.
## Settings layering
A run's settings come from four layers, and each sets values over the one below. Tool settings, chat options and run
options all resolve this way, once, when the run starts.
### One value, four layers
Follow the `find_companies` limit of the [getting started](https://weboost-at.github.io/Typeshift/guide/getting-started.md) agent up through the layers:
```mermaid
flowchart BT
code["1. The code's defaults: Limit = 5"] --> agent["2. The agent's code: WithLimit(3)"]
agent --> stored["3. The stored settings: Limit = 2"]
stored --> run["4. The run: Limit = 1"]
run --> tool["find_companies runs with Limit = 1"]
```
**1. The code's defaults.** `FindCompaniesSettings` initializes `Limit` to 5, so a record made with its parameterless
constructor holds 5. This layer sets every member, so a tool always receives a whole record.
**2. The agent's code.** `ResearchAgent` calls `ConfigureFindCompanies().WithLimit(3)`, so every run of the agent
searches with 3 unless a layer above sets another limit. Its chat and run options, such as `WithTemperature(0.2f)`,
live here too. Because this layer is reviewed, compiled code, it may set any settable member, configurable or not.
**3. The stored settings.** `ConfigureAsync(42, ...)` stores `Limit = 2` and `Temperature = 0.4f`, so every run with
settings id 42 searches with 2, at a temperature of 0.4. `IncludeInactive` cannot be stored, because
`ResearchAgent.Settings` has only the configurable members.
**4. The run.** A run that passes `FindCompanies = new() { Limit = 1 }` in its settings searches with 1, for that run
only. It leaves `Temperature` null, so the stored 0.4 holds.
### Precedence
The highest layer that sets a value wins:
| Layer | Set by | May set | Holds for |
|---|---|---|---|
| 4. The run | The `settings` argument of `RunAsync` and the other runs | Configurable members | That run |
| 3. Stored | `Settings` under the run's settings id: `ConfigureAsync`, `SaveSettingsAsync` | Configurable members | Every run with that settings id |
| 2. The agent's code | `Configure{Tool}()`, `WithChatOptions()`, `WithRunOptions()`, `GetInstructions()` | Every settable member | Every run of the agent |
| 1. The code's defaults | The settings record's parameterless constructor; the run options' defaults | Every member | Every run |
Because the layers resolve when a run starts, settings saved during a run apply from the next run on.
### Null values and nested records
A layer either replaces a value below it or leaves it alone. It never merges, so a configurable member that is itself
a record or a class is replaced as a whole.
::: warning No layer clears a value
Null keeps the layer below, so neither stored settings nor a run can take away a temperature the agent's code set.
:::
### The model and the instructions
The model resolves through the layers like any other value. It stays as the code sets it unless
`Configurable(o => o.ModelId)` lets the layers above choose it, so stored settings can change the model only when the
agent's code allows that. Its provider is a value of its own, so a layer that sets only `Model` keeps the provider of
the layer below.
Stored or run `Instructions` replace the text that `GetInstructions()` returns. Context providers and plan mode then
append to whatever instructions the layers resolved.
### The mode of a conversation
A run with a conversation id continues in the mode (`Default`, `Plan` or `Auto`) that the conversation's last run
ended in. That mode overrides the agent's code and the stored settings, but not the run's own settings. A run whose
own settings name a mode switches the conversation to it, and so does an approved plan
([The three modes](https://weboost-at.github.io/Typeshift/guide/tool-approval.md#the-three-modes)).
### Sub-agents and handoffs
Sub-agents and handoffs keep the settings id, but not the run's settings. A sub-agent runs with its own settings stored
under the caller's settings id, in the caller's mode. An agent handed off to answers with its own stored settings under
the same id. Run settings apply only to the agent the run started on ([Multiple agents](https://weboost-at.github.io/Typeshift/guide/multi-agent.md)).
## Registering a store
Register the store in the `AddTypeshift()` chain:
```cs
var services = new ServiceCollection();
services.AddTypeshift()
.WithStore()
.WithChatClientProvider();
services.AddKeyedSingleton("openai", new CannedChatClient());
await using var provider = services.BuildServiceProvider(validateScopes: true);
```
`InMemoryTypeshiftStore` keeps everything in memory and serves tests and samples. For your database, write a store of
your own: [Implementing a store](https://weboost-at.github.io/Typeshift/guide/implementing-a-store.md) covers its methods, its versions and the conversation
summaries it reads.
`WithStore()` registers each store interface your class implements, so an `ITypeshiftStore` takes one call. A
store of settings and a store of conversations take one call each.
::: warning The store is a singleton
`WithStore()` registers the store as a singleton, so one instance serves every run at the same time. It cannot
take scoped services, such as a `DbContext`: [Register your store](https://weboost-at.github.io/Typeshift/guide/implementing-a-store.md#register-your-store)
shows how to build one around that.
:::
Without a store, agents run from their code alone. A run that passes neither a settings id nor a conversation id
works, tools included. Every call that loads or saves, such as a run with either id or `ConfigureAsync`, throws an
`InvalidOperationException` that names `WithStore()` and the interface no registered store implements.
## Agent settings
A settings id stands for whatever you decide: a customer, a team, a user, or a settings profile of your own. Under one
settings id, each agent has its own document, because each agent's `Settings` is its own kind, so a store keys
documents by id and kind together.
`ConfigureAsync` loads an agent's settings, applies your change and saves them. When another save came in between, it
loads them again and reapplies your change, ten tries in all. If the tenth save loses too, it throws the store's
`StoreConflictException`:
```cs
await support.ConfigureAsync(settingsId, settings => settings with
{
Provider = "anthropic",
Model = "claude-sonnet-5-5",
Instructions = "Answer in German.",
SearchTickets = settings.SearchTickets with { Limit = 20 }
});
```
A settings form does it in two steps: it loads the settings with their version and saves them back with it. The
version tells the store which state the form loaded, so the save throws a `StoreConflictException` when someone saved
in between ([Save with a version](https://weboost-at.github.io/Typeshift/guide/implementing-a-store.md#save-with-a-version) explains the number):
```cs
var stored = await support.LoadSettingsAsync(settingsId);
try
{
await support.SaveSettingsAsync(settingsId, stored.Data with { Instructions = null }, stored.Version);
}
catch (StoreConflictException)
{
stored = await support.LoadSettingsAsync(settingsId);
}
```
Saving `Instructions = null` removes the stored instructions, so runs with this settings id use those of
`GetInstructions()` again. After a conflict, the sample loads the settings again, so you can show the user the current
values before they save.
## The stored JSON
What is stored is the record's JSON: camelCase, with the tools nested under `"tools"` by name and nulls left out.
```json
{
"provider": "anthropic",
"model": "claude-sonnet-5-5",
"instructions": "Answer in German.",
"tools": {
"search_tickets": {
"limit": 20
}
}
}
```
An agent whose `WithSchemaVersion(n)` raised its schema version above 1 also writes `"$schemaVersion": n` first.
`System.Text.Json` writes and reads the same JSON, so an endpoint that binds `SupportAgent.Settings` accepts and returns
the JSON the store keeps. A stored member that cannot be read, because its value has the wrong type or its name is
unknown, counts as missing, so the layer below applies. When a configurable member is itself a record or a class, a
stored value replaces the whole object, so members missing from its stored JSON take that type's defaults, not the
values of the layer below.
::: warning A renamed member loses its stored value
A configurable member that is renamed or removed reads as missing, so its stored value no longer applies. When that
matters, raise the agent's schema version with `WithSchemaVersion(2)` and migrate the documents of the older
schema version ([Migrate older settings](https://weboost-at.github.io/Typeshift/guide/implementing-a-store.md#migrate-older-settings)).
:::
## Related
* [Tool settings](https://weboost-at.github.io/Typeshift/guide/tool-settings.md): declaring a tool's settings and opening members to the layers.
* [Implementing a store](https://weboost-at.github.io/Typeshift/guide/implementing-a-store.md): the store's methods, its versions and conversation summaries.
* [Generated members](https://weboost-at.github.io/Typeshift/guide/generated-members.md#settings): the `Settings` record a store keeps.
* [Running agents](https://weboost-at.github.io/Typeshift/guide/running-agents.md#run-settings): passing a settings id and run settings.
* [Conversations](https://weboost-at.github.io/Typeshift/guide/conversations.md): what a run stores in a conversation, and resuming it.
* [Registration](https://weboost-at.github.io/Typeshift/guide/registration.md#the-id-type): the id type and the other chain calls.
---
---
url: https://weboost-at.github.io/Typeshift/guide/conversations.md
description: >-
Store a conversation under an id, resume it after a crash, steer and watch a
run, and compact a long one.
---
# Conversations
Pass a conversation id to a run, and Typeshift stores every step under it. The next run picks up where the last one
ended, even after a crash.
## Tracking a conversation
```cs
var conversationId = 7;
await support.RunAsync("Where is my refund?", conversationId: conversationId);
await support.RunAsync("When will it arrive?", conversationId: conversationId);
```
The first run starts the conversation and the second resumes it. Typeshift puts the stored messages ahead of the new
input, so the model sees the whole conversation. Because Typeshift sends the history itself, it never passes the
conversation id on to the provider. Settings ids and conversation ids are independent, and a run may pass either or
both. A conversation does not remember a settings id, so pass it to every run of the conversation that should use the
stored settings, the run that answers an approval included.
Typeshift never creates ids ([the id type](https://weboost-at.github.io/Typeshift/guide/registration.md#the-id-type)). You choose the conversation id, for
example the key of your own record of the conversation, and pass the same id to every run of that conversation.
::: danger Check the caller before you pass a conversation id
Typeshift resumes whichever conversation it is given, so deciding who may see one is your app's job. Check the caller
against your own record of the conversation, as you would for any other id from a request.
:::
## What is stored
A conversation is a list of entries, appended as they happen: each run's input, every model response with its model and
token usage, the tool results of each round of calls, and an entry that ends the run and records its mode. Your store
receives each entry through `AppendConversationAsync`
([Implementing a store](https://weboost-at.github.io/Typeshift/guide/implementing-a-store.md#append-to-a-conversation)).
Each entry is a `ConversationEntry`: the name of its agent in `Agent`, its `Messages` and the time it was stored in
`At`, plus the `Model` and `Usage` of a model response, and `EndsRun` on the entry that ends a run. `IsSummary` is true
on the entry [compaction](#compaction) appends when it summarizes, which is how your code and your store tell a summary
apart ([Start from the last summary](https://weboost-at.github.io/Typeshift/guide/implementing-a-store.md#start-from-the-last-summary)).
An entry serializes itself with `Serialize()` and `ConversationEntry.Deserialize(json)`, and JSON libraries handle it as
it is. `System.Text.Json` writes and reads that same JSON, and Newtonsoft.Json writes it as a string, so a store or an
API can pass entries on with either library and no settings.
```cs
var store = provider.GetRequiredService();
var entries = await store.LoadConversationAsync(conversationId, ct);
Console.WriteLine($"{entries.Count} entries, {entries.Sum(entry => entry.Usage?.TotalTokenCount)} tokens");
```
Runs that fail or stop early are stored too:
* A failed run stores an entry with an `ErrorContent` that ends the run, then rethrows the exception to your code.
* A streaming run that fails, is cancelled, or that your code stops reading early stores the updates that had arrived.
* A cancelled run is stored too, because Typeshift stores without the run's cancellation token.
* A run cut off by a crash has no ending entry, and [resuming](#resuming) continues it.
When the store fails to save, the failure is logged and recorded on the run's span, and the run still returns its
answer. Whenever a run loads the conversation, it leaves tool calls that never got a result and error messages out of
what the model sees, because providers reject them.
Some providers, such as local servers with an OpenAI-compatible API, number their tool calls per response, so a later
call can get an id the conversation holds already. Typeshift renames such a call, as `call_1_2`, before it stores or
runs it, so the stored entries, `ApprovalRequests` and the tool's result all carry the new id.
## Resuming
Each model response is stored as it arrives, and tool results before the next model call, so an app that crashes
mid-run loses at most the calls in flight. `ResumeAsync` continues the conversation's interrupted run from its last
stored step, without new input:
```cs
var resumed = await support.ResumeAsync(interrupted, settingsId);
```
On a conversation whose last run ended, `ResumeAsync` throws an `InvalidOperationException`. After a restart, the
entries tell you which conversations to resume: when the last entry that `LoadConversationAsync` returns has
`EndsRun` false, a crash cut that run off. A new run with input works on an interrupted conversation too: it starts
from the interrupted run's steps and adds its own input.
::: warning A tool can be called twice
A call interrupted while its tool ran is left out on resume. The model decides again and may call the tool a second
time, so make tools with side effects safe to repeat.
:::
## One run at a time
Within a process, a conversation has one run at a time. Starting a second while the first goes on throws an
`InvalidOperationException`, so [steer](#steering) the running one or wait until it ends. A run that starts while
another is still loading the conversation waits until that run has begun. Across processes, keeping to one run per
conversation is up to your app.
## Live runs
`ConversationTracker` lists the runs with a conversation id that are going on in this process, each with its
conversation, the agent that answers, its model and when it started:
```cs
var tracker = provider.GetRequiredService();
foreach (var run in tracker.Runs)
Console.WriteLine($"{run.Agent} is answering conversation {run.Conversation} on {run.Model}");
```
`WatchRunsAsync` tells a UI when that list changes, so a list of running conversations needs no polling. It yields the
runs going on, then each run as it starts and again as it ends, with `Ended` set, until you cancel its token.
Cancelling ends the loop with an `OperationCanceledException`, so catch it where you stop watching. It works without
`WithRealtimeTracking()`:
```cs
await foreach (var run in tracker.WatchRunsAsync(ct))
Console.WriteLine($"Conversation {run.Conversation} is {(run.Ended ? "idle" : "running")}");
```
## Steering
`TrySteer` hands new input to a conversation's run while it works, for example when a user adds a request halfway
through. The model receives it before its next call, and the run does not end before then. The input is stored like
any other. When the run had asked for approval, its requests close once it answers the input, as they would for a new
message ([When a request closes](https://weboost-at.github.io/Typeshift/guide/tool-approval.md#when-a-request-closes)).
`TrySteer` returns false when no run of the conversation is going on in this process, or when that run is already
ending. Start a run with the input instead:
```cs
var input = "Send me the invoice too.";
if (!tracker.TrySteer(conversationId, input))
await support.RunAsync(input, settingsId, conversationId);
```
To stop a run, cancel its token. It is stored up to where it stopped.
## Live updates
With `WithRealtimeTracking()`, Typeshift publishes the updates of runs as they happen, and `WatchAsync` reads one
conversation's updates until you cancel its token, which ends the loop with an `OperationCanceledException`, as with
`WatchRunsAsync`. A streaming run publishes its updates as they stream; a run that does not stream publishes each
model response and tool result as it stores it. A UI can forward them over SignalR, for example:
```cs
await foreach (var update in tracker.WatchAsync(conversationId, ct))
Console.Write(update.Text);
```
Text, reasoning and usage updates are merged and published at most every 100 ms, and other updates go out at once. A
run never waits for a watcher, so a watcher that falls more than 256 updates behind loses the oldest ones. You can
change both limits with `RealtimeTrackingOptions`:
```cs
services.AddTypeshift()
.WithRealtimeTracking(new RealtimeTrackingOptions
{
PublishInterval = TimeSpan.FromMilliseconds(250),
MaxPendingUpdates = 1024
});
```
Without `WithRealtimeTracking()`, `WatchAsync` throws an `InvalidOperationException`.
::: warning Live updates stay in their process
A watcher sees only the runs of its own process. When your app runs in several processes, watch in the process that
runs the conversation.
:::
## Compaction
A coding agent can make hundreds of model calls in one run, and its history soon outgrows the model's context.
Compaction shrinks the history each call sees. It is off until `WithMaxInputTokens` sets a limit, and the other
[run options](https://weboost-at.github.io/Typeshift/guide/builders.md#runoptionsbuilder) let such a run go on for long enough: here `WithMaxToolIterations(500)`
raises the limit on rounds of tool calls to 500. The agent also starts in [plan mode](https://weboost-at.github.io/Typeshift/guide/tool-approval.md#plan-mode),
which has nothing to do with compaction:
```cs
WithRunOptions()
.WithMode(AgentMode.Plan)
.WithMaxToolIterations(500)
.WithMaxInputTokens(150_000)
.WithSummaryModel("claude-haiku-5-5")
.Configurable(r => r.Mode)
.Configurable(r => r.MaxToolIterations)
.Configurable(r => r.SummaryModel);
```
Before every model call, Typeshift estimates the call's input tokens at about four characters per token, corrected by
what the provider reported for the run's last call, or for the conversation's last call before the run. When the
estimate reaches the maximum, it shrinks the history to `WithTargetInputTokens`, half the maximum unless set:
1. It clears the results of the oldest tool calls the model has already read. It keeps the calls, so the model knows
what it did and can call again.
2. If that is not enough, it summarizes the oldest turns with the [summary model](https://weboost-at.github.io/Typeshift/guide/models.md#the-summary-model), the
run's own unless you name another.
The instructions and the latest call of a tool named `write_todos`, if the agent offers one, always stay, and a cut
never separates a tool call from its result.
Every later model call of the same run starts from the history as compaction left it, with the same results cleared
and the same summary.
The stored conversation keeps everything. Compaction appends an entry of its own, with `IsSummary` set to true, that
holds the summary and the messages it kept after it. Later runs start from that entry instead of summarizing again, and
read nothing before it: the entry also keeps the conversation's mode, its [approvals for the
conversation](https://weboost-at.github.io/Typeshift/guide/tool-approval.md#approving-for-the-conversation) and the background agents still working.
## Related
* [Running agents](https://weboost-at.github.io/Typeshift/guide/running-agents.md): passing a conversation id to a run.
* [How a run works](https://weboost-at.github.io/Typeshift/guide/running-agents.md#how-a-run-works): where a run resumes and stores its steps.
* [Implementing a store](https://weboost-at.github.io/Typeshift/guide/implementing-a-store.md#append-to-a-conversation): the store methods that keep conversations.
* [Builders](https://weboost-at.github.io/Typeshift/guide/builders.md#runoptionsbuilder): the run options of compaction and their defaults.
* [Models and providers](https://weboost-at.github.io/Typeshift/guide/models.md#the-summary-model): which model summarizes, and its provider.
* [Multiple agents](https://weboost-at.github.io/Typeshift/guide/multi-agent.md#handoff): handoffs, which need a conversation.
* [Serving a chat endpoint](https://weboost-at.github.io/Typeshift/guide/chat-endpoint.md): a conversation per chat, behind ASP.NET Core endpoints.
---
---
url: https://weboost-at.github.io/Typeshift/guide/chat-endpoint.md
description: >-
Run an agent from ASP.NET Core endpoints, stream its answer to a chat UI as
server-sent events, and answer tool approvals by their ids.
---
# Serving a chat endpoint
A chat UI talks to your agent through endpoints of your web app. The UI posts the user's message to a chat, gets the
answer back whole or as it streams, and posts the user's answers to the tool calls that wait for approval. This page
builds those endpoints with ASP.NET Core minimal APIs, on the [runs](https://weboost-at.github.io/Typeshift/guide/running-agents.md),
[conversations](https://weboost-at.github.io/Typeshift/guide/conversations.md) and [approvals](https://weboost-at.github.io/Typeshift/guide/tool-approval.md) the other pages describe.
## Post a message
Each chat is a conversation, and the chat's id in the route is its conversation id. The endpoint runs the agent in
that conversation, with the current customer's id as the [settings id](https://weboost-at.github.io/Typeshift/guide/running-agents.md#stored-settings), so the
customer's stored settings apply:
```cs
var chat = app.MapGroup("/chats/{chatId:int}")
.AddEndpointFilter();
chat.MapPost("/messages", async (
int chatId,
ChatInput input,
AgentHost support,
ICurrentCustomer customer,
CancellationToken ct) =>
{
var response = await support.RunAsync(
input.Text,
settingsId: customer.Id,
conversationId: chatId,
ct: ct);
return ChatReply.From(response);
});
```
* The host and `ICurrentCustomer` come from the request's scope, as
[Using an agent](https://weboost-at.github.io/Typeshift/guide/using-an-agent.md#inject-the-host-in-a-web-app) describes. `ICurrentCustomer` stands for a
scoped service of your app, filled from the request's authentication, so put your authentication on the group too,
for example with `RequireAuthorization()`.
* The `CancellationToken` is the request's, so the run is cancelled when the user leaves the page. A run that should
finish anyway takes no token.
* `ChatReply` is the JSON the UI gets back: the answer's text and the approvals it waits for.
The filter on the group checks that the chat belongs to the caller before any endpoint of the group runs, because
Typeshift resumes whichever conversation it is given
([Tracking a conversation](https://weboost-at.github.io/Typeshift/guide/conversations.md#tracking-a-conversation)). Map every endpoint that takes a chat id on
this group:
```cs
internal sealed class ChatOwnerFilter(IChats chats, ICurrentCustomer customer) : IEndpointFilter
{
public async ValueTask InvokeAsync(EndpointFilterInvocationContext context, EndpointFilterDelegate next)
{
var http = context.HttpContext;
var chatId = int.Parse((string)http.Request.RouteValues["chatId"]!);
return await chats.BelongsToAsync(chatId, customer.Id, http.RequestAborted)
? await next(context)
: TypedResults.NotFound();
}
}
```
`IChats` stands for your app's record of its chats, in which each chat names its customer. Another customer's chat
answers 404, as a chat that does not exist does, so the UI learns nothing about the chats of others.
## Return the answer
The endpoints answer with records of your own, so the UI's contract stays yours when Microsoft.Extensions.AI's types
change:
```cs
internal sealed record ChatInput(string Text);
internal sealed record ChatReply(string Text, IReadOnlyList Approvals)
{
public static ChatReply From(ChatResponse answer) =>
new(answer.Text, [.. answer.ApprovalRequests.Select(PendingApproval.From).OfType()]);
}
// What the UI shows to ask, and sends back with the user's answer
internal sealed record PendingApproval(
string RequestId,
string CallId,
string Tool,
IDictionary? Arguments)
{
public static PendingApproval? From(ToolApprovalRequestContent request) =>
request.ToolCall is FunctionCallContent call
? new(request.RequestId, call.CallId, call.Name, call.Arguments)
: null;
}
```
`ApprovalRequests` lists the calls that wait for approval ([A run that asks](https://weboost-at.github.io/Typeshift/guide/tool-approval.md#a-run-that-asks)).
`PendingApproval` keeps the tool and its arguments, which the UI shows when it asks, and the ids the UI sends back with
the user's answer. Show the arguments as the model sent them, because those are the arguments the call runs with.
## Stream the answer
The streaming endpoint runs the agent the same way, and hands its updates to
[`TypedResults.ServerSentEvents`](https://learn.microsoft.com/aspnet/core/fundamentals/minimal-apis/responses):
```cs
chat.MapPost("/messages/stream", (
int chatId,
ChatInput input,
AgentHost support,
ICurrentCustomer customer,
CancellationToken ct) =>
{
var updates = support.RunStreamingAsync(
input.Text,
settingsId: customer.Id,
conversationId: chatId,
ct: ct);
return TypedResults.ServerSentEvents(Events(updates));
});
```
Each update becomes a `text` event with the text it adds, and each approval request an `approval` event:
```cs
private static async IAsyncEnumerable> Events(IAsyncEnumerable updates)
{
await foreach (var update in updates)
{
if (update.Text is { Length: > 0 } text) yield return new SseItem