run-ai / genv

GPU environment and cluster management with LLM support
https://www.genv.dev
GNU Affero General Public License v3.0
445 stars 19 forks source link

genv remote llm fails to extract port number #61

Closed jangmarker closed 1 month ago

jangmarker commented 1 month ago

With genv 1.4.1 genv llm works for the multi-user use case. But genv remote llm doesn't seem to have picked up on the change:

$ genv remote -H <host> llm ps
HOST             MODEL       PORT    CREATED              EID     USER
<host>           llama2:7b/45893N/A     2 hours ago          2891    <user>

note how the model is reported as "llama2:7b/45893" with the port presumably being "N/A".

Correspondingly, the following happens:

$ genv remote -H <host> llm attach llama2:7b
Could not find LLM model 'llama2:7b'
$ genv remote -H <host> llm attach llama2:7b/45893
pulling manifest
Error: pull model manifest: file does not exist
razrotenberg commented 1 month ago

hi @jangmarker thanks for this issue! I fixed it and released Genv 1.4.2.

pls let me know if this issue is not resolved.

thanks again!

jangmarker commented 1 month ago

It's fixed, thanks a lot!