feat: select model with ui

[0xfraso]

Hi, I've been experimenting with ollama and found your awesome plugin. Being able to tinker with ollama within neovim is super fun!

My idea was to make users able to select model on the fly without having to edit their config. This PR aims to leverage ollama's REST API to make the most of its capabilities, and not just from its CLI tools.

This said, users can make use of the introduced GenSelectModel to set it on the fly (might be worth in the future to implement some sort of file caching for persistent settings).

I introduced plenary.nvim as dependency to this project (that might be worth if each and every plugin call will make use of the exposed REST API), specifically to use plenary.curl for HTTP requests.

I think this could be super useful for those who want to have a dedicated machine to host their ollama server and not having it locally.

Best regards Francesco

[David-Kunz]

Hi @0xfraso ,

Thank you for this nice PR! At the moment, I try not to introduce another dependency (plenary) and try to keep it as extensible as possible (e.g. if users want to switch the command).

I noticed that it's also possible to use

ollama list

to get a list of all installed models. But I'm not sure how useful this information is. Usually you would configure the model either globally or per prompt, but not on the fly. Or do you disagree with that assumption?

Thank you for the discussion and best regards, David

[0xfraso]

Hi @David-Kunz !

Since the list/model selection feature is not yet implemented (you have to set the model manually if you don't use the default one), It could be worth implementing both the API call and the CLI feature you mentioned.

I preferred to go for the latter because to me is much cleaner and extensible to parse a JSON response rather than extracting the model names with some kind of regex (or maybe I'm wrong). Additionally, It could be worth using plenary.curl (that almost everybody have probably already installed) to make calls to a remote server.

Cheers Francesco

[wishuuu]

Hi @David-Kunz !

I also think introducing plenary.nvim sounds like sounds like good idea. As @0xfraso mentioned, most users will have it already installed, as it is dependency of many popular plugins.

Next to pros mentioned by Francesco, it will also allow redesigning command execution part of script to use plenary,job so it will work on Windows system too. Right now there is problem with execution that I'll describe soon as new Issue. Unfortunately, the tech stack at my job prevents me from using exclusively Linux 😞, so it would be great if we could manage to make gen.nvim working on Windows too.

Usually you would configure the model either globally or per prompt, but not on the fly. Or do you disagree with that assumption?

Some models might be trained better on particular technology, users working with multiple technologies might use option to change models dynamically. I'm not sure if that's the case, I'm far from being an expert in LLM, so feel free to correct me if I'm wrong.

Best regards Oskar

[leafo]

Call me old-school but my vote is for keeping the plugin simple and not adding a bunch of specific integrations (and dependencies) for all the different methods of invoking LLMs currently out and will exist in the future.

gen.nvim already uses a simple shell command to interact with the LLM: I think this is very good separation of concerns and is in line with the unix philosophy. I actually use this tool with ChatGPT, and the integration took 2 seconds: I already had a simple command line script that hits their API and outputs the result that I was able to set as the command option and was good to go.

My suggestion: Lua is flexible, I vote for supporting functions in the configuration for the various phases of interacting with the model. There can be a setup function, and then a command function. This will allow those with specific needs, like setting up docker images or picking models, to delegate those tasks to those functions instead of adding even more configuration parameters.

[David-Kunz]

Hi @leafo ,

Yes, I also try to keep this plugin as minimal and extensible as possible.

FYI: Due to some changes with Ollama, we had to switch to HTTP-based communication, hence if you want to use ChatGPT with gen.nvim, you now need to configure

init: function(ops)
        --  initialize it, empty function if not needed
       end,
command: function(ops)
  return 'your command to run, probably curl with OpenAI API'
end,
json_response = false -- otherwise we'll try to extract the `.response` property

@0xfraso , I decided not to allow users to select the model, this should be done in the configuration phase, where users have access to ollama list anyway.

Thank you and best regards, David

[David-Kunz]

Hi @0xfraso ,

After others also wanted this feature, I implemented it (without plenary to reduce dependencies): https://github.com/David-Kunz/gen.nvim/pull/44

Best regards, David

[0xfraso]

Hi @David-Kunz , I'm glad that the whole project switched to a HTTP based approach, imho this is the most scalable way to make this project grow and for sure It was the whole point of this PR.

On the other hand I didn't mean to bother anyone with unnecessary dependencies (even though I think that any Neovim user might already have it installed as dependency from other plugins), and just introduce a very simple feature targeting ollama's REST APIs. In both cases, curl would have been used as backend, either natively or within a library, but I guess It does matter.

Best Regards, Francesco

[David-Kunz]

Hi @0xfraso ,

Thank you for your comment. Yes, many users probably have plenary installed, it's also one of my dependencies. Nevertheless, the less dependencies we have, the better. And it's not really needed, since it's uses curl anyway and we don't need all those other features.

I also think the HTTP approach is quite good, although it comes with a bit less extensibility (parsing the output is quite specific). But all in all we're on a good track :)

Thank you again for your great participation and best regards, David

[David-Kunz]

P.s. you didn't bother anyone, your PR was of great help!