Ollama models can not be loaded, if not predefined (even if API of ollama stays the same)

[BananaAcid]

Before submitting your bug report

• [X] I believe this is a bug. I'll try to join the Continue Discord for questions
• [X] I'm not able to find an open issue that reports the same bug
• [X] I've seen the troubleshooting guide on the Continue Docs

Relevant environment info

- OS: Windows 11
- Continue: v0.7.54
- Ollama: 0.1.16 in WSL2/Debian

- models testet, but do not work: orca2:latest, orca2:7b, dolphin-mixtral:latest, codellama:latest, mario:latest (created by modelfile as described in the ollama readme)

Description

There does not seem to exist a way to use any model, that is not predefined in Continue.

I tried adding the following to the config.json

    {
      "title": "dolphin-mixtral",
      "provider": "ollama",
      "model": "dolphin-mixtral:latest",  // also dolphin-mixtral-latest does not work, https://ollama.ai/library/dolphin-mixtral
      "apiBase": "http://localhost:11434"
    }

As well as the the others the same way. Using model codellama-7b works.

To reproduce

$ ollama pull dolphin-mixtral
$ ollama list

NAME                    ID              SIZE
codellama:7b            8fdf8f752f6e    3.8 GB
codellama:latest        8fdf8f752f6e    3.8 GB
dolphin-mixtral:latest  4b33b01bf336    26 GB

Log output

No response

[sestinj]

@BananaAcid despite the yellow underline indicating error (we will fix this I realize it's extremely misleading), you should still be able to use that model.

If that still isn't working, let me know, this would be much different bug!

[BananaAcid]

@sestinj could you elaborate on how the model line above should look and how the the model_roles.default should look?

[sestinj]

@BananaAcid model roles is no longer necessary and won't have an effect, it is just leftover from old config.json files.

Otherwise, your config above looks perfect. If you choose one of the models that are listed in the dropdown (that do not give yellow underline), then we will map those options onto their respective tag names for Ollama. If you give an option that is not in the dropdown, we will just pass that exact string to Ollama, so as long as you use a valid tag name, you can ignore the yellow line and it should work.

Obviously a bit confusing, it's on my mind to reorganize this very soon

[BananaAcid]

Perfect, yes it works.

Reloading the dropdown does not seem to work, but restarting vscode after each change in the config.json does seem to do the trick.

[pjv]

@sestinj when you get around to working on this, it would be handy if it were possible to make a non-standard ollama model be the default (rather than continue always defaulting to GPT-4).

[sestinj]

@pjv do you mean that when you reload the window it should remember which model you were using last, or something else?

And is this pre-release, or main version you are using?

[pjv]

do you mean that when you reload the window it should remember which model you were using last, or something else?

Yeah, remembering would be great. Alternatively the ability to specify a default via settings (gui or json) like I guess used to be possible with config.json would be fine too.

And is this pre-release, or main version you are using?

I’m using v0.7.58. I’ve configured a few recent Ollama models viaconfig.json. They are working fine but regardless of what order I put them into config.json in, whenever the window is reloaded or a new one opened, continue always defaults to GPT-4.

[sestinj]

@pjv Ok great, I've made the change to persist this info (as well as saving any current session history). Will be available later today in v0.7.60 (pre-release)