Header menu logo Nao

IStreamingLlmProvider Type

Optional capability a provider implements when it can return the completion incrementally (token by token) instead of buffering the whole response. `onChunk` is invoked on the calling task for each delta as it arrives; the returned `CompletionResult` is the fully aggregated completion (identical in shape to `CompleteAsync`), so a caller that streams to a UI and also needs the final text requires no extra bookkeeping.

Instance members

Instance member Description

this.StreamAsync

Full Usage: this.StreamAsync

Parameters:
Returns: Task<CompletionResult>
Modifiers: abstract
arg0 : Conversation
arg1 : CompletionOptions
arg2 : CompletionChunk -> unit
Returns: Task<CompletionResult>

Type something to start searching.