search

SciSharp / LLamaSharp

A C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.
https://scisharp.github.io/LLamaSharp
MIT License
2.7k stars 349 forks source link

Specify file open permissions needed as read only. #943

Closed scritch1sm closed 1 month ago

scritch1sm commented 1 month ago

Default open behavior looks to be read/write. Resulting in failed file open of model in environments with strict read-only file system settings)

martindevans commented 1 month ago

This is a good idea, and I'll merge it after the tests have run. Thanks!

However, I'm not sure if it will entirely fix the problem. The FileStream there is not actually used to read the model, all we're doing is opening and immediately closing the file. This acts as a sanity check that the file exists and is readable before we pass off to llama.cpp to do the actual work of reading the file.

The model is really loaded with llama_load_model_from_file (a couple of lines down). I don't know whether or not that loads the file with read-only permissions.

scritch1sm commented 1 month ago

makes sense, but FWIW in my particular use case (i can't share too much of the particulars as it came up during work), it did resolve an issue. the filesystem in question simply didn't allow opening for write and would throw an exception here, causing loading the model to fail outright here on this line. I compiled this in and was able to successfully load the model in context and the application moved on with life. I then ran into OTHER issues, but that's an unrelated other story. =)

On Tue, Oct 8, 2024 at 6:10 AM Martin Evans @.***> wrote:

This is a good idea, and I'll merge it after the tests have run. Thanks!

However, I'm not sure if it will entirely fix the problem. The FileStream there is not actually used to read the model, all we're doing is opening and immediately closing the file. This acts as a sanity check that the file exists and is readable before we pass off to llama.cpp to do the actual work of reading the file.

The model is really loaded with llama_load_model_from_file (a couple of lines down). I don't know whether ot not that loads the file with read-only permissions.

— Reply to this email directly, view it on GitHub https://github.com/SciSharp/LLamaSharp/pull/943#issuecomment-2399805038, or unsubscribe https://github.com/notifications/unsubscribe-auth/BKHRFLQQFKKYCL33QZPGG7DZ2PKTTAVCNFSM6AAAAABPRILU56VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOJZHAYDKMBTHA . You are receiving this because you authored the thread.Message ID: @.***>