Status
What is built, which test covers each behavior, and what is still open. Open items live here, never as TODOs in code.
Slices
The rewrite built the API the owner approved in ten slices, one at a time, and all ten are done:
- Toolsets and tools: generated functions, overrides, tool lists, the analyzer.
- Agents, registration,
AgentHostruns. - Settings store, id type.
- Conversations: checkpoints, resume, live updates, steering.
- Approval, rules, plan and auto modes.
- Middleware, context providers.
- Telemetry, compaction.
- Sub-agents, handoff.
- MCP, built-in toolsets.
- Native AOT: the console sample, the package consumer.
Behavior coverage
A number is the rule's row in v0's behavior table, where that rule still holds. Rows marked "new" came with the rewrite.
| # | Behavior | Test |
|---|---|---|
| 1 | A tool's schema holds only the model-facing parameters | ToolSchemaTests.Given_Parameters_The_Call_Fills_They_Stay_Out_Of_The_Schema |
| 3 | Overrides apply over the settings below; members that are not configurable have no override | ToolOverridesTests |
| 4 | A call receives its arguments, its settings, the run's services and the call's token; the result reaches the model as JSON | ToolInvocationTests.Given_A_Scripted_Call_The_Tool_Receives_Its_Arguments, ..._Given_Settings_The_Tool_Runs_With_Them, ..._Given_Two_Calls_Of_One_Run_Both_Resolve_From_The_Runs_Scope, ..._Given_A_Call_The_Tool_Receives_The_Calls_Token, ..._Given_A_Scripted_Call_The_Result_Reaches_The_Model_As_Json |
| 5 | A missing required argument fails the call; an absent optional one takes its default | ToolInvocationTests.Given_A_Missing_Required_Argument_The_Call_Fails_With_An_ArgumentException, ..._Given_An_Absent_Optional_Argument_The_Tool_Takes_Its_Default |
| 6 | A struct parameter defaulting to default | ToolInvocationTests.Given_A_Struct_Parameter_Defaulting_To_Default_The_Tool_Runs_Without_It, ToolSchemaTests.Given_A_Struct_Parameter_Defaulting_To_Default_Its_Schema_Carries_The_Default |
| new | A project that only documents types gets a module, so a referencing host registers its docs (D55) | ToolsetGenerationTests.Given_A_Schema_Annotations_Class_And_No_Toolsets_Or_Agents_A_Module_Registers_Its_Docs, AgentGenerationTests.Given_A_Reference_Of_Docs_Only_The_Interceptor_Registers_Its_Module |
| new | A tool returning IAsyncEnumerable<T> gives the model its items as one array; under AOT the list needs JSON metadata (D45) | ToolInvocationTests.Given_A_Tool_That_Streams_Its_Results_The_Model_Receives_Them_As_One_Array, DiagnosticTests.Given_A_Stream_Whose_List_No_Context_Lists_Under_Native_Aot_TSH027_Is_Reported_On_Its_Tool |
| 7 | A nullable reference parameter is typed nullable | ToolSchemaTests.Given_A_Nullable_Reference_Parameter_It_Is_Typed_As_Nullable |
| 8 | The schema matches AIFunctionFactory's | ToolSchemaTests.Given_The_Same_Method_The_Schema_Matches_AIFunctionFactory_Without_Descriptions |
| new | A recursive parameter type's $refs point into its parameter, as AIFunctionFactory's do | ToolSchemaTests.Given_A_Recursive_Parameter_Type_The_Schema_Matches_AIFunctionFactory |
| 9 | Docs become descriptions | ToolSchemaTests.Given_Param_Docs_They_Become_The_Parameter_Descriptions, ..._Given_No_WithDescription_The_Summary_Is_The_Tool_Description |
| 10 | Tool names are unique | DiagnosticTests.Given_Two_Tools_Mapping_To_One_Member_TSH007_Is_Reported_On_The_Second, ..._Given_Two_Toolsets_With_A_Tool_Of_One_Name_TSH007_Is_Reported_On_The_Second |
| new | A tool name is unique across the host, two references' tools included (TSH007) | DiagnosticTests.Given_A_Tool_Named_Like_One_Of_A_Reference_TSH007_Is_Reported_On_The_Own, ..._Given_Two_References_With_A_Tool_Of_One_Name_TSH007_Is_Reported |
| new | Tool names derive from the method, without Async (D13) | ToolSchemaTests.Given_An_Async_Method_Without_WithName_The_Tool_Is_Named_Without_The_Suffix, ToolNamesTests |
| new | A call without services fails, saying how to run the tool | ToolInvocationTests.Given_No_Services_The_Call_Fails_Saying_How_To_Run_It |
| new | Every shape the generator cannot handle is a diagnostic, and the generated code holds no error | DiagnosticTests, DiagnosticTests.Given_Any_Mistake_The_Generated_Code_Holds_No_Error |
| new | Parameters the model never sees need no <param> | ToolsetGenerationTests.Given_Parameters_The_Model_Never_Sees_Their_Missing_Docs_Are_Suppressed |
| 11 | AddTypeshift() registers the project's and the references' toolsets and agents, once | AddTypeshiftTests |
| 12 | A call the generator did not intercept throws, saying what to check | AddTypeshiftTests.Given_A_Call_The_Generator_Did_Not_Intercept_It_Fails_Saying_What_To_Check |
| 13 | A run gets the agent's code options; run settings override the configurable ones, null keeps the code | AgentRunTests.Given_A_Run_The_Model_Receives_The_Agents_Code_Options, ..._Given_Run_Settings_They_Override_The_Configurable_Options, ..._Given_Run_Settings_That_Leave_An_Option_Null_The_Agents_Code_Holds |
| 14 | An agent offers its toolsets' tools, included in pick order or all but the excluded, then its own | AgentRunTests.Given_The_Agents_Toolsets_The_Model_Is_Offered_Their_Tools_Then_Its_Own, ..._Given_Exclude_Every_Other_Tool_Is_Offered_In_Declaration_Order |
| 15 | Tool settings layer: code defaults, Configure{Tool}(), the run's overrides | AgentRunTests.Given_Configure_The_Tool_Runs_With_The_Agents_Settings, ..._Given_Run_Overrides_Of_A_Tool_They_Apply_Over_The_Agents_Settings |
| new | A failed tool call reaches the model with its exception's message (D53) | AgentRunTests.Given_A_Failing_Tool_Call_The_Model_Reads_Why_It_Failed |
| new | A run's tools resolve from the scope its host came from (D25) | AgentRunTests.Given_Two_Tool_Calls_Of_A_Run_Both_Resolve_From_The_Hosts_Scope |
| new | Messages and updates carry the agent's name (D26) | AgentRunTests.Given_A_Run_Its_Messages_Carry_The_Agents_Name_As_Their_Author, ..._Given_A_Streaming_Run_Its_Updates_Carry_The_Text_And_The_Agents_Name |
| new | Run options limit the tool loop and may be configurable (D32) | AgentRunTests.Given_Run_Settings_Of_A_Run_Option_The_Tool_Loop_Stops_After_It |
| new | Calls the host approves run in the next run, within its own limit (D24, D32) | ToolApprovalTests.Given_A_Run_That_Asked_At_Its_Limit_The_Next_Run_Runs_The_Approved_Call_Within_Its_Own |
| new | A run asks the chat client provider for the provider and model of code, stored settings or the run; the request names the model alone (D56) | AgentRunTests.Given_Run_Settings_They_Override_The_Configurable_Options, ..._Given_A_Provider_And_A_Model_The_Request_Names_Only_The_Model, ..._Given_Options_Of_Its_Own_The_Created_Client_Asks_The_Agents_Provider, StoredSettingsTests.Given_Stored_Settings_A_Run_With_Their_Id_Applies_Them |
| new | The provider id type types the alias, the Provider members and the core services (D56) | AgentGenerationTests.Given_A_Provider_Id_Type_The_Alias_The_Settings_And_The_Core_Services_Use_It |
| new | A public agent's configurable model needs a provider id type as accessible (TSH006, D6) | DiagnosticTests.Given_A_Public_Agent_Of_An_Internal_Provider_Id_Type_TSH006_Is_Reported_On_The_Model_Pick |
| new | Structured output and the raw path | AgentRunTests.Given_A_Result_Type_The_Run_Returns_The_Models_Json_Typed, ..._Given_Resolved_Options_The_Created_Client_Runs_The_Agent |
| new | A run without a model or a provider fails, saying how to fix it | AgentRunTests.Given_No_Model_The_Run_Fails_Saying_How_To_Set_One, AddTypeshiftTests.Given_No_Chat_Client_Provider_A_Run_Fails_Naming_The_Registration |
| new | Disposing a run's client leaves the provider's client alone | AgentRunTests.Given_A_Run_The_Providers_Client_Outlives_It |
| new | An agent's own tools (AddTool in its constructor) are offered after its toolsets' tools, with settings, approval and the analyzer's tool rules (D57) | OwnToolsTests, AgentGenerationTests.Given_Tools_Of_The_Agents_Own_They_Are_Generated_In_It_And_Offered_After_The_Toolsets, ..._Given_Tools_Of_The_Agents_Own_The_Generated_Code_Compiles_Without_Warnings, DiagnosticTests.Given_An_Agents_Tool_Of_An_Instance_Method_TSH003_Is_Reported_On_The_Argument, ..._Given_A_Sub_Agent_With_A_Tool_Of_Its_Own_That_Needs_Approval_TSH024_Is_Reported_On_The_Use |
| new | An agent's own tool needs a unique name only among the agents its standing approvals reach or come from: handoffs, directly or through others, and sub-agents, a referenced project's agent included (D33, D57) | DiagnosticTests.Given_Two_Agents_With_Tools_Of_One_Name_And_No_Handoff_TSH007_Is_Not_Reported, ..._Given_A_Handoff_Between_Agents_With_Tools_Of_One_Name_TSH007_Is_Reported_On_Both_Tools, ..._Given_A_Sub_Agent_With_A_Tool_Named_Like_Its_Callers_TSH007_Is_Reported_On_Both_Tools, ..._Given_Agents_Handing_Off_To_Each_Other_With_Tools_Of_One_Name_TSH007_Is_Reported_Once_On_Each_Tool, ..._Given_Handoffs_Reaching_An_Agent_With_A_Toolset_Tool_Of_One_Name_TSH007_Is_Reported_On_The_Own_Tool, ..._Given_A_Handoff_To_A_References_Agent_With_A_Tool_Of_One_Name_TSH007_Is_Reported_On_The_Own_Tool, ..._Given_A_Handoff_To_A_References_Agent_With_A_Tool_Of_A_Toolsets_Name_TSH007_Is_Reported_On_The_Agent, ..._Given_An_Agents_Tool_Named_Like_Its_Toolsets_But_For_Underscores_TSH007_Is_Reported_On_The_Own_Tool, ..._Given_An_Agents_Tool_Named_Like_A_Toolsets_It_Does_Not_Use_TSH007_Is_Not_Reported |
| new | The XML docs of a parameter's type describe its properties; the module registers a marked class's docs (D55) | AddTypeshiftTests.Given_Xml_Docs_On_A_Parameters_Type_Its_Schema_Describes_Its_Properties, ToolsetGenerationTests.Given_A_Schema_Annotations_Class_The_Module_Registers_Its_Docs, ..._Given_A_Schema_Annotations_Class_That_Is_Not_Partial_The_Module_Leaves_It_Out |
| new | Toolsets of referenced assemblies: stamps, settings, JSON contexts (D7, D45) | AgentRunTests.Given_A_Toolset_Of_A_Referenced_Assembly_Its_Tool_Runs_With_Its_Settings, AddTypeshiftTests.Given_A_Json_Context_In_A_Toolsets_Assembly_Tools_Ask_It_Before_The_Defaults, AgentGenerationTests |
| new | An agent's output is generated again only when what it offers changes | IncrementalTests |
| 16 | Stored settings apply under their id, run settings over them, tool overrides included | StoredSettingsTests.Given_Stored_Settings_A_Run_With_Their_Id_Applies_Them, ..._Given_Stored_And_Run_Settings_The_Runs_Apply_Over_The_Stored, ..._Given_Stored_Tool_Overrides_The_Tool_Runs_With_Them_Over_The_Agents_Settings |
| 17 | Saves are guarded by versions; ConfigureAsync retries until it wins, ten times at most (D41) | StoredSettingsTests.Given_A_Version_Another_Save_Replaced_Saving_Throws_A_StoreConflictException, ..._Given_A_Stale_Version_Saving_Throws_A_StoreConflictException, ..._Given_A_Save_Between_Load_And_Save_Configuring_Applies_The_Change_Again_And_Wins, ..._Given_A_Save_Between_Every_Load_And_Save_Configuring_Gives_Up_After_Ten_Attempts |
| new | A store keeps each agent's settings as JSON under its kind, so one id holds a document per agent (D38) | StoredSettingsTests.Given_One_Id_For_Two_Agents_The_Store_Keeps_The_Json_Of_Each_Under_Its_Kind |
| new | The docs' EF Core store checks and writes each save in one statement, and a save that changed no row (an insert at version 0 or an update of a stale version) throws StoreConflictException and leaves the settings as they were (D41) | samples/Typeshift.Samples.Docs, AgentSettings scenario, run by CI |
| new | The in-memory store passes VerifyAsync through each of its interfaces, a store that breaks a rule (kinds under one id, conflicts, the order of entries, the last summary) fails with a message naming it, and a store that is both fails on either interface's rules whichever it is called through (D58) | StoreExtensionsTests |
| 18 | Without a store, loading or saving throws naming WithStore<T>() (D43) | StoredSettingsTests.Given_No_Store_A_Run_With_A_Settings_Id_Fails_Naming_WithStore |
| new | A store keeps settings, conversations or both; WithStore<T>() registers each interface it implements, and a missing one is named (D38, D43) | StoredSettingsTests.Given_A_Store_Of_Settings_Alone_A_Run_With_Their_Id_Applies_Them, ..._Given_A_Store_Of_Conversations_Alone_Loading_Settings_Fails_Naming_ISettingsStore, ConversationStoreTests.Given_A_Store_Of_Settings_Alone_A_Run_With_A_Conversation_Fails_Naming_IConversationStore, ..._Given_A_Store_Of_Settings_And_One_Of_Conversations_A_Run_Uses_Each |
| new | A store that implements neither interface for the build's id type is TSH028 (D11, D38) | DiagnosticTests.Given_A_Store_Of_Conversations_Of_Another_Id_Type_TSH028_Is_Reported_On_WithStore, ..._Given_A_Store_Of_Settings_Alone_Of_The_Builds_Id_Type_TSH028_Is_Not_Reported |
| 19 | Settings of every storable kind read back equivalent; nulls left out; mistyped members read as null | SettingsJsonTests |
| new | A record struct stores like a record; another struct is TSH012 (D19) | SettingsJsonTests.Given_Settings_Of_Every_Kind_They_Read_Back_Equivalent, DiagnosticTests.Given_A_Configurable_Struct_That_Is_Not_A_Record_TSH012_Is_Reported_On_The_Pick |
| new | System.Text.Json writes and reads the stored shape (D44) | SettingsJsonTests.Given_Settings_System_Text_Json_Writes_What_The_Store_Keeps, ..._Given_Stored_Json_System_Text_Json_Reads_The_Settings |
| new | A conversation entry serializes as it is: System.Text.Json in its stored shape, Newtonsoft.Json as that JSON in a string (D44) | ConversationEntryJsonTests |
| new | Kinds and schema versions (D39, D40) | SettingsJsonTests.Given_WithStorageKind_And_WithSchemaVersion_The_Settings_Carry_Them, ..._Given_A_Schema_Version_Newer_Than_The_Codes_Reading_Throws_A_JsonException |
| new | The JSON carries the schema version as "$schemaVersion", left out at 1, so a store keeps no schema version; every reader refuses a newer one (D40) | SettingsJsonTests.Given_WithSchemaVersion_The_Json_Carries_The_Schema_Version, ..._Given_Schema_Version_1_Settings_Without_Values_They_Are_Stored_As_An_Empty_Object, ..._Given_A_Schema_Version_Newer_Than_The_Codes_System_Text_Json_Throws_A_JsonException, ..._Given_A_Schema_Version_That_Is_No_Whole_Number_Reading_Throws_A_JsonException, ConversationEntryJsonTests.Given_An_Entry_Of_A_Newer_Schema_Version_Reading_Throws_A_JsonException |
| new | A kind is unique, two referenced projects' agents included (TSH022, D39) | DiagnosticTests.Given_Two_Agents_Of_One_Class_Name_TSH022_Is_Reported_On_The_Second, ..._Given_Two_References_With_Agents_Of_One_Kind_TSH022_Is_Reported |
| new | The id type reaches the aliases and the typed members (D11) | AgentGenerationTests.Given_An_Id_Type_The_Aliases_And_The_Host_Members_Use_It |
| new | A public agent of a less accessible id type is TSH006, and its host members stay internal (D6) | DiagnosticTests.Given_A_Public_Agent_Of_An_Internal_Id_Type_TSH006_Is_Reported_On_The_Agent |
| 20 | A conversation's next run starts from its stored messages; the provider never sees its id | ConversationRunTests.Given_A_Conversation_A_Second_Run_Starts_From_The_First, ..._Given_A_Conversation_The_Provider_Never_Receives_Its_Id |
| 21 | Each step is stored as it happens; the last entry ends the run (D50) | ConversationRunTests.Given_A_Run_Its_Input_Responses_And_Tool_Results_Are_Stored_Step_By_Step, ..._Given_A_Run_Only_Its_Last_Entry_Ends_It, ..._Given_A_Model_Response_Its_Entry_Carries_The_Model_And_The_Agent |
| 22 | A failed run ends with its error, which the next run leaves out; a stream left early is stored (D46) | ConversationRunTests.Given_A_Failing_Run_It_Ends_With_Its_Error_And_The_Exception_Surfaces, ..._Given_A_Failed_Run_The_Next_Run_Leaves_Out_Its_Error, ..._Given_A_Stream_Left_Early_The_Run_Ends_With_The_Updates_That_Arrived |
| 23 | A store that fails to append does not fail the run | ConversationStoreTests.Given_A_Store_That_Fails_To_Append_The_Run_Still_Answers |
| new | Resuming continues an interrupted run without the call in flight (D50) | ConversationRunTests.Given_An_Interrupted_Run_Resuming_Continues_From_Its_Last_Step_Without_The_Call_In_Flight, ..._Given_A_Conversation_Whose_Last_Run_Ended_Resuming_Fails |
| new | One run per conversation in a process, claimed before the run loads it; the tracker lists it (D50) | ConversationRunTests.Given_A_Run_Going_On_A_Second_Run_Of_The_Conversation_Fails, ..._Given_A_Run_Going_On_The_Tracker_Lists_It, ..._Given_A_Run_That_Failed_To_Begin_The_Next_Run_Of_The_Conversation_Runs, ConversationStoreTests.Given_A_Run_Loading_Its_Conversation_A_Second_Run_Of_It_Fails_Once_The_First_Began, ..._Given_A_Run_Loading_Its_Conversation_That_Fails_To_Begin_A_Second_Run_Of_It_Runs |
| new | Steering reaches the next model call, is stored, and keeps the run going (D51) | ConversationRunTests.Given_Steering_Input_The_Run_Answers_It_Before_It_Ends, ..._Given_Steering_Input_During_The_Answer_To_Steering_The_Run_Answers_Both_In_Order, ..._Given_Steering_Input_It_Is_Stored_With_The_Conversation |
| new | Run watchers hear the runs going on, then each start and end (D54) | ConversationRunTests.Given_A_Run_Watcher_It_Hears_The_Run_Start_And_End, ..._Given_A_Run_Going_On_A_Run_Watcher_Hears_It_Once_Then_Its_End |
| new | Watchers receive streamed updates (D47) | ConversationRunTests.Given_A_Watcher_It_Receives_The_Updates_Of_A_Streaming_Run |
| new | Text streamed within the publish interval reaches watchers merged, ahead of the next call (D47) | ConversationRunTests.Given_Text_Within_The_Interval_A_Watcher_Receives_It_Merged_Ahead_Of_The_Next_Call |
| new | A run that does not stream publishes each response and tool result to watchers (D47) | ConversationRunTests.Given_A_Watcher_It_Receives_Each_Step_Of_A_Run_That_Does_Not_Stream |
| new | The raw path names its conversation in ChatOptions.ConversationId | ConversationRunTests.Given_A_Conversation_Id_In_The_Options_The_Raw_Path_Stores_The_Run |
| new | Concurrent calls on one created client each keep their own iteration limit (D29, D32) | AgentRunTests.Given_Two_Calls_On_One_Created_Client_Each_Keeps_Its_Own_Limit |
| new | A call that requires approval ends the run with its request; the next run's answer runs or rejects it (D24) | ToolApprovalTests.Given_A_Call_That_Needs_Approval_The_Run_Ends_With_Its_Request_And_It_Does_Not_Run, ..._Given_An_Approval_The_Next_Run_Of_The_Conversation_Runs_The_Call, ..._Given_A_Rejection_The_Model_Receives_Its_Reason_And_The_Call_Does_Not_Run, ..._Given_A_Streaming_Run_The_Host_Receives_The_Request_And_Its_Approval_Runs_The_Call, ..._Given_A_Streaming_Run_Left_At_Its_Request_The_Approval_Still_Runs_The_Call |
| new | The stored request is the call that runs; stray and second answers change nothing (D24) | ToolApprovalTests.Given_An_Answer_Carrying_Other_Arguments_The_Call_The_Conversation_Stored_Runs, ..._Given_An_Answer_To_No_Stored_Request_The_Run_Throws_And_Nothing_Runs, ..._Given_A_Second_Answer_To_A_Request_The_First_Stands, ..._Given_An_Answered_Approval_Later_Runs_Send_The_Call_Once_With_Its_Result, ..._Given_An_Input_Carrying_A_Request_Of_Its_Own_And_Its_Approval_It_Throws_And_Nothing_Runs |
| new | Only the requests the last run left open can be answered: an answer to a request a later run passed by throws, runs nothing and approves nothing for the conversation (D24, D60) | ToolApprovalTests.Given_A_Request_A_Later_Run_Passed_By_An_Answer_To_It_Throws_And_Runs_Nothing, ..._Given_A_Request_A_Later_Run_Passed_By_Its_Approval_For_The_Conversation_Holds_Nothing |
| new | An answer to a closed request throws ApprovalClosedException, naming each request and why it is closed, before the run stores anything or calls the model, also through a handoff and without a conversation (D60) | ToolApprovalTests.Given_Answers_To_Closed_Requests_The_Exception_Names_Each_With_Why_It_Is_Closed, ..._Given_An_Answer_To_A_Closed_Request_Nothing_Of_The_Run_Is_Stored, ..._Given_A_Streaming_Run_Answering_A_Closed_Request_It_Throws, ..._Given_No_Conversation_An_Answer_Throws_Asking_For_The_Conversation, HandoffTests.Given_A_Closed_Request_Of_The_Agent_Handed_To_An_Answer_Throws_And_Frees_The_Conversation |
| new | A run that answers steering input after asking for approval closes its requests and leaves them out of ApprovalRequests, also when it answers by asking again; a model call that fails first leaves them open, and a background agent's answer closes none (D60) | ToolApprovalTests.Given_Steering_The_Model_Answers_After_A_Request_The_Run_Returns_No_Request, ..._Given_A_Request_Its_Steered_Run_Passed_By_An_Answer_To_It_Throws_And_Runs_Nothing, ..._Given_A_Streaming_Run_Steered_After_Its_Request_An_Answer_To_It_Throws, ..._Given_A_Steered_Run_That_Failed_Before_The_Model_Answered_Its_Request_Stays_Open, ..._Given_Steering_The_Model_Answers_By_Asking_Again_Only_The_New_Request_Is_Returned, ..._Given_Steering_The_Model_Answers_By_Asking_Again_The_New_Requests_Approval_Runs_It, SubAgentTests.Given_A_Request_While_A_Background_Sub_Agent_Works_The_Run_Returns_It_And_Its_Approval_Runs |
| new | Watchers see only the approval requests left to the host (D24, D60) | ToolApprovalTests.Given_A_Watcher_It_Sees_Only_The_Requests_Left_To_The_Host |
| new | A call id the conversation holds already is renamed, so a provider that reuses ids across runs still gets its later approvals run (D59) | ToolApprovalTests.Given_A_Provider_That_Reuses_Call_Ids_Across_Runs_A_Later_Approval_Runs_Its_Call, ..._Given_A_Provider_That_Reuses_Call_Ids_Across_Streaming_Runs_A_Later_Approval_Runs_Its_Call, ..._Given_An_Unanswered_Request_Whose_Id_A_Later_Call_Reuses_That_Calls_Approval_Runs_It, ..._Given_An_Unanswered_Streamed_Request_Whose_Id_A_Later_Call_Reuses_That_Calls_Approval_Runs_It, ..._Given_A_Call_Left_At_The_Limit_Whose_Id_A_Later_Call_Reuses_Each_Call_Goes_Out_With_Its_Result |
| new | Calls next to one that requires approval wait with it; without a conversation a run that asks fails | ToolApprovalTests.Given_A_Call_Needing_Approval_Next_To_One_That_Does_Not_Only_The_First_Waits_And_Both_Run, ..._Given_No_Conversation_A_Call_Needing_Approval_Fails_Naming_The_Tool |
| new | Standing approvals and rules decide before the host; a rejection wins (D24) | ToolApprovalTests.Given_An_Approval_For_The_Conversation_A_Later_Call_Of_The_Tool_Runs_Without_Asking, ..._Given_A_Rule_That_Approves_The_Call_Runs_Without_Asking, ..._Given_A_Rule_That_Rejects_The_Model_Receives_Its_Reason, ..._Given_One_Rule_That_Approves_And_One_That_Rejects_The_Rejection_Wins, ..._Given_A_Call_Needing_Approval_A_Rule_Sees_It_With_The_Runs_Mode, ..._Given_A_Rule_Answering_One_Call_And_The_Host_Another_Both_Run_Once_The_Host_Answered, ..._Given_A_Streaming_Run_And_A_Rule_That_Approves_No_Request_Reaches_The_Host |
| new | Auto mode runs what requires approval unless a rule rejects it (D52) | ModeTests.Given_Auto_Mode_A_Call_Needing_Approval_Runs_Without_Asking, ..._Given_Auto_Mode_A_Rule_That_Rejects_Still_Wins, ..._Given_Auto_Mode_The_Calls_It_Approves_Count_Toward_The_Runs_Limit |
| new | Plan mode offers the read-only tools and exit_plan_mode; the approved plan continues in the chosen mode (D52) | ModeTests.Given_Plan_Mode_Only_The_Read_Only_Tools_And_Exit_Plan_Mode_Are_Offered, ..._Given_Plan_Mode_The_Instructions_Ask_For_A_Plan, ..._Given_Plan_Mode_Presenting_The_Plan_Ends_The_Run_With_A_Request_That_Carries_It, ..._Given_An_Approved_Plan_The_Run_Continues_In_The_Chosen_Mode, ..._Given_A_Rejected_Plan_The_Model_Receives_The_Feedback_And_Stays_In_Plan_Mode |
| new | A conversation keeps its mode unless a run names one (D52) | ModeTests.Given_A_Conversation_A_Run_Without_A_Mode_Continues_In_The_Mode_The_Last_Run_Ended_In, ..._Given_A_Run_Naming_A_Mode_It_Leaves_The_Conversations_Mode |
| new | Run middleware wraps runs in order, may change the messages, and may answer in place of the model (D30) | RunMiddlewareTests |
| new | Streaming runs pass through run middleware, which reads the response once the stream ended (D30) | RunMiddlewareTests.Given_A_Streaming_Run_The_Middleware_Reads_The_Response_Once_The_Stream_Ended, ..._Given_A_Streaming_Run_Middleware_That_Answers_Is_Streamed, ..._Given_A_Streaming_Run_Left_Early_The_Run_Ends_With_What_Arrived |
| new | Tool middleware wraps calls in order and may answer in place of the tool (D30) | ToolMiddlewareTests |
| new | Context providers add instructions, unstored messages ahead of the input, and tools (D31) | ContextProviderTests |
| new | A run is an invoke_agent span around its chat and tool spans, with usage and errors (D36) | TelemetryTests |
| new | Runs on one provider and model share one instrumented client, whatever client instances the provider returns (D36) | TelemetryTests.Given_Two_Runs_On_A_Provider_Client_Per_Call_Their_Chat_Spans_Come_From_One_Source, ..._Given_Two_Runs_On_One_Provider_Client_Their_Chat_Spans_Come_From_One_Source |
| new | Compaction clears the oldest read results, then summarizes the oldest turns, by calibrated estimate (D32) | CompactionTests |
| new | The summary runs on the run's provider and model unless the agent or the settings name others (D56) | CompactionTests.Given_No_Summary_Model_The_Runs_Model_Summarizes, ..._Given_Clearing_Is_Not_Enough_The_Oldest_Turns_Are_Summarized, ..._Given_A_Summary_Provider_It_Serves_The_Summary_Model |
| new | A summary is stored, and later runs start from it (D32) | CompactionTests.Given_A_Summary_Later_Runs_Start_From_It |
| new | A run loads its conversation from the last summary on, which keeps the mode, the standing approvals and the background calls still working (D24, D32, D33, D52) | ConversationStoreTests.Given_A_Summary_The_Next_Run_Never_Loads_The_Whole_Conversation, ..._Given_A_Summary_The_Store_Loads_It_And_The_Entries_After_It, ToolApprovalTests.Given_An_Approval_For_The_Conversation_The_Summary_Keeps_It_For_Later_Runs, ModeTests.Given_A_Compaction_In_Plan_Mode_Its_Summary_Keeps_The_Mode, SubAgentTests.Given_A_Compaction_While_A_Background_Sub_Agent_Works_Its_Summary_Keeps_The_Call, ..._Given_A_Summary_That_Left_A_Background_Sub_Agent_Working_The_Next_Run_Hears_It_Was_Lost |
| new | A steered run's later pass starts from the summary its first pass made (D32, D51) | CompactionTests.Given_A_Steered_Streaming_Run_Its_Later_Pass_Starts_From_The_Summary |
| new | A run's first estimate is calibrated by the conversation's last reported call (D32) | CompactionTests.Given_A_Report_Of_The_Last_Run_The_Next_Runs_First_Call_Compacts_By_It |
| new | Summarizing keeps the latest call of a tool named write_todos with its result (D32, D37) | CompactionTests.Given_Summarized_Turns_The_Latest_Todo_List_Stays_With_Its_Result |
| new | A sub-agent is a tool of its description and one input, answering in a fresh context in its caller's mode (D33) | SubAgentTests |
| new | Plan mode offers the sub-agents, which plan with their read-only tools and no exit_plan_mode (D33, D52) | SubAgentTests.Given_A_Caller_In_Plan_Mode_Its_Sub_Agents_Are_Offered, ..._Given_A_Caller_In_Plan_Mode_Its_Sub_Agent_Is_Offered_Only_Its_Read_Only_Tools |
| new | A sub-agent's calls that need approval are rejected unless something approves them (D33) | SubAgentTests.Given_A_Sub_Agent_In_The_Default_Mode_A_Call_That_Needs_Approval_Is_Rejected |
| new | A sub-agent inherits the standing approvals of its caller's conversation (D24, D33) | SubAgentTests.Given_A_Tool_Approved_For_The_Callers_Conversation_The_Sub_Agents_Call_Of_It_Runs |
| new | A background sub-agent's answer joins the run, which waits for it; a lost one is reported on resume (D33) | SubAgentTests.Given_A_Background_Sub_Agent_The_Caller_Works_On_And_Receives_Its_Answer_Before_It_Ends, ..._Given_An_Interrupted_Run_With_A_Background_Sub_Agent_The_Model_Hears_It_Was_Lost |
| new | A handoff continues the run with the other agent, which answers the conversation's later runs (D34) | HandoffTests |
| new | A run that starts while a handoff passes the conversation on is refused, and the agent handed to answers (D34, D50) | HandoffTests.Given_A_Run_Starting_While_A_Handoff_Passes_On_The_Conversation_The_Agent_Handed_To_Answers, ..._Given_A_Run_Starting_While_A_Streamed_Handoff_Passes_On_The_Conversation_The_Agent_Answers, ..._Given_A_Handoff_Whose_Agent_Fails_Before_It_Begins_A_Run_Waiting_For_The_Conversation_Runs, ..._Given_A_Handoff_Back_To_The_First_Agent_In_The_Same_Run_The_First_Agent_Answers |
| new | Sub-agents and handoff targets are checked at compile time (TSH023, TSH024, TSH026) | DiagnosticTests |
| new | A referenced sub-agent is checked against its module stamp (TSH023, TSH024; D33) | DiagnosticTests.Given_A_Sub_Agent_Of_A_Reference_Without_A_Description_TSH023_Is_Reported_On_The_Use, ..._Given_A_Sub_Agent_Of_A_Reference_Offering_A_Tool_That_Needs_Approval_TSH024_Is_Reported_On_The_Use |
| new | MCP servers' tools follow the agent's own, read-only or behind approval; a clash or no provider fails (D35) | McpTests |
| new | A reference with agents built with another provider id type is TSH021 (D56) | DiagnosticTests.Given_A_Reference_With_Agents_Built_With_Another_Provider_Id_Type_TSH021_Is_Reported |
| new | A reference without agents has no id type to clash (D11) | DiagnosticTests.Given_A_Reference_Of_Toolsets_Only_Built_With_Another_Id_Type_TSH021_Is_Not_Reported |
| new | Under Native AOT, a tool type no context lists is TSH027 (D45) | DiagnosticTests.Given_A_Tool_Type_No_Context_Lists_Under_Native_Aot_TSH027_Is_Reported_On_Its_Tool |
| new | Tools, settings, conversations, streaming and structured output run published as Native AOT | samples/Typeshift.Samples.Console, run by CI |
| new | A consumer of the package, not of the projects, generates and registers its agents | samples/Typeshift.Samples.PackageConsumer, run by CI |
Known gaps
- Run limit across approval rounds. The rounds the approval layer continues, after a rule, a standing approval or auto mode approved calls, share the run's
MaxToolIterations, so a round that runs tools before it asks again may run one round of tools past it. Calls the host approves are not affected: they run in the next run. - Compaction's estimate. About four characters make a token, corrected by the provider's report of the run's last call, or of the conversation's last call before it, so only a conversation's first call trusts the estimate alone. The summary model reads results cut to 2,000 characters.
- Uses inside a referenced project. TSH007 follows the handoffs and sub-agents of your project's agents into a referenced project's agent, but not that agent's own uses, which no stamp lists. An own tool of yours and one of an agent the referenced agent hands off to are not compared, in either project.
- One conversation through unrelated hosts. An app that runs one conversation id through the hosts of two agents that never hand off to each other shares their standing approvals, and no diagnostic sees it.
Related
- Decisions: why the built behavior is shaped the way it is.
- Architecture: where each behavior lives in the code.