razrotenberg
commented
2 months ago hi @jangmarker, thanks again for reaching out with this issue! I fixed this and released version 1.4.1.
now, instead of finding the listening port using /proc/pid (and getting access denied in the case of multiple Linux users), we keep the listening port as part of the environment name behind the scenes.
this way, all users can find the listening port by parsing the environment name.
this of course is done transparently by the genv llm cli command.
I'm closing this issue, lmk if you encounter it again.
thanks again! :raised_hands:
jangmarker
From the documentation it should be possible to serve an LLM as one user "user1" and then have other users, e.g. "user2", attach to it via genv llm attach modelname.
However, in practice this fails on Linux hosts because genv, when run as "user2", cannot determine the ollama port from the ollama process id if "user1" hosts the model. This is because /proc/processid/fd is only readable by the user who owns the process, in this case "user1".
A workaround is to punch holes in Linux' process isolation, but that's far from ideal. Ideally, genv could track the ollama port besides the process id and make it available to other users, or solve this differently altogether.