search

dusty-nv / jetson-containers

Machine Learning Containers for NVIDIA Jetson and JetPack-L4T
MIT License
1.89k stars 416 forks source link

Cannot run ollama use command "docker run" on Jetson AGX Orin 64GB #483

Closed leeyoshinari closed 2 months ago

leeyoshinari commented 2 months ago

when I run with "jetson-containers run $(autotag ollama)", it works. but when I run with "docker run --runtime nvidia -it --rm --network=host dustynv/ollama:r36.2.0", it reoprts error.

Couldn't find '/root/.ollama/id_ed25519'. Generating new private key. Error: could not create directory mkdir /root/.ollama: file exists

I enter inside the docker image, 'id_ed25519' exists and is in '/root/.ollama'. '/root/.ollama' is a soft link to '/data/models/ollama'.

If I want to run ollama in the background, what shoud i do? pls~

dusty-nv commented 2 months ago

@leeyoshinari can you try docker pull dustynv/ollama:r36.2.0 , I am able to run the same docker run --runtime nvidia -it --rm --network=host dustynv/ollama:r36.2.0 command as you without issue.

You can also override the model cache mount point using --volume /host/models:/root/.ollama . otherwise with no mounts in your docker run command, you will need to re-download the models the next time the ollama server container shuts down.

leeyoshinari commented 2 months ago

solve it, thank you!