Open Cherry opened 3 months ago
I also having the same issue
const answer = await c.env.AI.run("@cf/meta/llama-3-8b-instruct", {
messages: [{ role: "user", content }],
});
shows the following error
No overload matches this call.
The last overload gave the following error.
Argument of type '"@cf/meta/llama-3-8b-instruct"' is not assignable to parameter of type 'BaseAiImageToTextModels'
What's a good way to fix this? Without completely changing the function overloading that's going on?
Function overloading is always a pain, but something like:
env.AI.run<"text">(...)
and then pinning the arguments to "text" models, or "image" models, etc. might make this a little easier to pin types, and remain backwards compatible, if the generic is optional. @DaniFoldi might have some other ideas - he's more of a TS wizard than me.
Hmm, my first guess would be giving NoInfer
a go, but unfortunately it is relatively new (5.4) and I'm pretty sure workers-types aims to support 4.8+, and it is declared as type NoInfer<T> = intrinsic
in lib.es5, so it's out.
I'll try to come up with an example, but our past discussion with James was about the const generic parameter he mentioned above, so models are categorized by goals/options.
we used generics to fix this, I'll quickly test and send a PR soon!
@Dhravya is this on a branch you could point us to? curious what your solution looks like!
Any updates on this issue? If the fix looks like it's still a ways out, could be good to update docs somehow
Today, the Workers AI types rely heavily on function overloads to specify arguments for different models. This unfortunately results in very difficult to debug types, and poor DX.
As an example with a more simplified non-AI class: https://tsplay.dev/Nan9ym
Throughout this during development, you get very little assistance with auto-complete, and the errors you get back, are extremely complex and hard to parse:
Let's use some real AI examples. These can all be found in https://tsplay.dev/wQB41N - scroll down to line 255 for examples.
Invalid model names
This errors with:
This is because I typo'd the model name and it should be
@hf/thebloke/neural-chat-7b-v3-1-awq
.Bad inputs
Here I've typo'd
messages
tomessage
, but the error is still just:Bad options
Here I mis-typed
extraHeaders
to be a boolean instead of an object, and my error is still:Essentially, due to the function overload complexity here, it seems that the errors surfaced to users from these types are really not that helpful. People are confused about this regularly in the Discord.