Recursive delegation

This guide is non-normative. It shows practice; it creates no semantics and states no guarantee. Evaluation owns recursion, concurrency owns tasks, tools and execution own a Flow run used as a Tool, and history owns how nested histories relate.

The pattern

Recursive language models handle inputs too large or too tangled for one model call by letting the model delegate: it splits the problem, asks sub-calls about the parts, sometimes writes code to inspect the input, and combines what comes back. Each sub-call may delegate again.

Flow needs no mechanism of its own for this. Each piece is an ordinary form:

Sub-calls are recursion

A delegating step asks the model to either answer or split, and recurses on the parts with depth + 1:

pub type Step {
    Answer(Text),
    Split(List<Text>),
    Program { source: Text, input: Text },
}

flow solve(question: Text, context: Text, depth: Int, limits: Limits) -> Result<Text, Problem> !tool = {
    let step = model.decide(question, context, depth) ? Deciding
    match step {
        Answer(text) => Ok(text),
        Split(parts) => {
            guard depth < limits.max_depth else { return Err(TooDeep(depth)) }
            let answers = task.map_limit(parts, limits.fan_out, flow(part) = solve(question, part, depth + 1, limits))
            combine(question, answers, depth)
        },
        Program { source, input } => execute(question, source, input, depth, limits),
    }
}

depth is ordinary data: the program decides what happens at the limit, and the model sees it as part of its input. Each part's answer is a Result, so a failed sub-call is a value the combining step can mention rather than a crash. map_limit keeps the fan-out bounded and returns answers in input order (concurrency).

The tasks form a tree that history records exactly: each sub-call's model call has an id under its task, such as root/2/1#1 (history), so a viewer can show which answer came from which branch.

Model-written code is a nested run

When the model writes a program to examine its input, the host's run Tool checks and runs it as a separate Flow run (execution). std's runner module declares it:

// std@v1/runner, implemented by the host
@tool pub type Run
pub type RunState { Completed(Text), Faulted(Text), Paused { run: Run, pending: List<Pending> } }

@tool pub flow check(source: Text) -> Result<Unit, List<Diagnostic>>
@tool pub flow run(source: Text, input: Text) -> Result<RunState, RunError>
@tool pub flow resume(run: Run, call: Text, reply: Text) -> Result<RunState, RunError>

Input and output cross as canonical JSON in Text, because the outer program cannot know the inner program's types. The delegating step uses it like any other Tool:

flow execute(question: Text, source: Text, input: Text, depth: Int, limits: Limits) -> Result<Text, Problem> !tool =
    match runner.check(source) {
        Err(Failed(diagnostics)) => solve(question, explain(source, diagnostics), depth + 1, limits),
        Err(problem) => Err(Running(describe(problem))),
        Ok(_) => match runner.run(source, input) {
            Ok(Completed(output)) => Ok(output),
            Ok(Faulted(message)) => solve(question, "The program faulted: ${message}", depth + 1, limits),
            Ok(Paused { .. }) => Err(Running("the program paused")),
            Err(problem) => Err(Running(describe(problem))),
        },
    }

What that buys:

• The inner program is checked before it runs. Diagnostics come back as data the model can read and fix.
• Its faults are values. A nested fault comes back as Faulted and never faults the outer run.
• It has its own history, whose Start names the outer run and the call that started it. The outer history holds one call and one reply per operation, and replaying the outer run serves those replies without running the inner program again.
• It reaches the outside only through Tools, bound by the host, by default with the outer run's bindings.

Budgets stay with the host

Tokens, money, wall-clock time and the number of runs a delegation tree may start are limits on the world around the run. Flow has no budget mechanism, and a library should not imitate one. The host enforces those limits around each Tool call; a call it refuses comes back as NotRun, which the program handles like any other problem. The depth and fan_out arguments above are the program's own shape, not a spending policy.

A library, not a feature

Recursive delegation ships as an ordinary library outside std. A sketch of its surface:

// github.com/example/delegate/delegate.flow
pub type Limits { pub max_depth: Int, pub fan_out: Int }

pub type Problem {
    Deciding(ToolProblem<model.ModelError>),
    Combining(ToolProblem<model.ModelError>),
    Running(Text),
    TooDeep(Int),
}

pub flow ask(question: Text, context: Text, limits: Limits) -> Result<Text, Problem> !tool =
    solve(question, context, 0, limits)

A caller imports it and calls delegate.ask like any other function; the host binds the model and runner Tools the library reaches before the run starts (tools). Someone who wants a different splitting rule, a different combining step, or no nested runs at all copies the library and changes it: it is a few dozen lines of ordinary Flow.