microsoft / vscode-prompt-tsx

MIT License
44 stars 3 forks source link

Improvements to `renderPrompt` #112

Open TylerLeonhardt opened 1 month ago

TylerLeonhardt commented 1 month ago

renderPrompt uses an IChatEndpointInfo:

export interface IChatEndpointInfo {
    /**
     * The maximum number of tokens allowed in the model prompt.
     */
    readonly modelMaxPromptTokens: number;
}

but in VS Code API when we get a model, it looks like this:

    /**
     * Represents a language model for making chat requests.
     *
     * @see {@link lm.selectChatModels}
     */
    export interface LanguageModelChat {

        // ...

        /**
         * The maximum number of tokens that can be sent to the model in a single request.
         */
        readonly maxInputTokens: number;

        // ...
        }

So naturally in several places I have to make this silly object:

{ modelMaxPromptTokens: model.maxInputTokens }

and also pass in model for:

renderPrompt<P extends BasePromptElementProps>(endpoint: IChatEndpointInfo, ctor: PromptElementCtor<P, any>, props: P, model: LanguageModelChat, token: CancellationToken)

so I pass in the same information twice... and I have to massage things to work but we own both APIs.

I think this should be cleaned up to take in only a LanguageModelChat.

This also applies to PromptRenderer.

connor4312 commented 3 weeks ago

@joyceerhl @roblourens I think it makes sense to tweak this to match VS Code's API (such that interfaces are just assignable in most cases) now that it's all finalized. Any reason not to?

roblourens commented 3 weeks ago

Sounds good

joyceerhl commented 3 weeks ago

SGTM