search

dexaai / dexter

LLM tools used in production at Dexa
MIT License
75 stars 5 forks source link

Improve support for OpenAI-compatible LLM providers #28

Closed transitive-bullshit closed 6 months ago

transitive-bullshit commented 7 months ago

Trying to use dexter with non-OpenAI LLMs hosted by OpenRouter and seeing a few issues.

Cost warnings

CleanShot 2024-03-31 at 16 19 49@2x

This logs for every LLM call. Since there's no real standard here aside from OpenAI, it'd be nice to support this total_cost format as an optional parameter that we fallback to internally and skip the console.warn.

Will add more cases to this issue as I come across them.

transitive-bullshit commented 7 months ago

Adding a better screenshot of the issue

CleanShot 2024-04-02 at 20 37 32@2x