OpenAI API v1/models returns nothing that v1/internal/model/list does

[TiagoTiago]

Describe the bug

This is a bug, right?

Is there an existing issue for this?

• [X] I have searched the existing issues

Reproduction

Try it on http://127.0.0.1:5000/docs and compare the returned results; or access the API thru scripts or whatever.

The Continue VSCode/Codium extension for example is completely blind to what models I actually have when I try the auto-detect option; had to add things manually in it's config.json

Screenshot

No response

Logs

.

System Info

.

[egarma]

For me going to v1/models returns chatgpt3.5 even though I'm loading mixtral8x7b through exllamav2-hf loader

[egSat]

For me, no matter what model is loaded that /v1/models always returns the following:

{"object":"list","data":[{"id":"gpt-3.5-turbo","object":"model","created":0,"owned_by":"user"},{"id":"text-embedding-ada-002","object":"model","created":0,"owned_by":"user"}]}

[jonnysowards]

Looks like the controller is calling response = OAImodels.list_dummy_models() on line 148 of extensions/openai/script.py it should be calling response = OAImodels.list_models(). I am AFK at the moment but I should be able to open a PR to fix when I have a minute.

[ghost]

Is that really an issue? Seems more like intentional behavior to me. IMHO it is intentional that the endpoint here mocks OpenAI's model response. There is an internal endpoint for a list of (internal) models, which was also mentioned here. I've also asked myself why that is the case. Otherwise they could perhaps have simply combined them there?

[jonnysowards]

For true OpenAI api support it is necessary. When using continue.dev, it will query the OpenAI endpoint and add those 2 dummy models which do not work. With this changed it grabs the real models and adds them to continue.dev ready to go. So I am not 100% sure it is necessary as it can be worked around, just not sure why you would include the endpoint at all if it all it does is include dummy models.

[jonnysowards]

A reason it wouldn’t be an issue is if using an integration requires the OpenAI model names (the dummy ones) from that endpoint for the integration to work.

[github-actions[bot]]

This issue has been closed due to inactivity for 2 months. If you believe it is still relevant, please leave a comment below. You can tag a developer in your comment.

[TiagoTiago]

Unless that has actually been fixed (or there's a legit reason for the apparent misbehavior of the API), that bot has no business being here; can that useless thing be disabled please?