Closed leeyiding closed 3 months ago
francescolavra
commented
3 months ago I think your Nanos instance has less memory assigned to it (the default value used by Ops is 2 GB) than your application requires, and you need to give it more memory. You can set the amount of memory in the Ops configuration file by adding a "Memory" attribute in the "RunConfig" JSON object. Example to configure the instance with 4 GB:
"RunConfig": {
"Memory": "4G"
}
leeyiding
commented
3 months ago Great, your approach is correct. I have another question. The new version of ollama I just mentioned uses the subprocess method to enable the llama.cpp backend. Do you have any ideas to bypass it?
eyberg
commented
3 months ago you're going to need to reverse what they did there - you might consider opening an issue w/them on it
leeyiding
commented
3 months ago OK, I got it, thank you very much for your reply
Hello, I'm sorry to bother you again, but I encountered some problems in the process of building ollama applications, and I need your help.
The ollama version I use is 0.1.31, and my directory tree is as follows, ollama binaries can be downloaded from ollama-linux-amd64 and renamed to
ollama.The configuration file is as follows:
Step 1, run
./ollama servelocally to start a service, and then run./ollama pull qwen: 0.5bin another terminal to pull a model. Step 2, terminate the service started in the previous step, and then runops run ollama -c config.json -nto run Nanos, the program starts normally at this step Step 3, Call api on another terminal.curl http://localhost:11434/api/generate -d '{ "model": "qwen:0.5b","prompt": "Hello!" }'After completing the step 3, an error occurred. Here is the running log.As for why I don't use the latest version of ollama, it's because the llama.cpp backend was originally loaded through a dynamic dependency library, but in https://github.com/ollama/ollama/commit/58d95cc9bd446a8209e7388a96c70367cbafd653, it was changed to loading through subprocessing, which also means that the later version cannot run in Unikernel. In the commit description, it can be seen that the main purpose is to solve the problems of memory leaks and stability defects. So, is there any way to solve the above problem? Looking forward to your help.