search

containers / ramalama

The goal of RamaLama is to make working with AI boring.
MIT License
183 stars 21 forks source link

is my install busted or --nocontainer broken? #167

Open juhp opened 1 week ago

juhp commented 1 week ago

When I try run with --nocontainer I get a backtrace:

$ ramalama --nocontainer run tiny
Traceback (most recent call last):
  File "/usr/bin/ramalama", line 35, in <module>
    main(sys.argv[1:])
    ~~~~^^^^^^^^^^^^^^
  File "/usr/bin/ramalama", line 20, in main
    ramalama.init_cli()
    ~~~~~~~~~~~~~~~~~^^
  File "/usr/lib/python3.13/site-packages/ramalama/cli.py", line 82, in init_cli
    args.func(args)
    ~~~~~~~~~^^^^^^
  File "/usr/lib/python3.13/site-packages/ramalama/cli.py", line 325, in run_cli
    model.run(args)
    ~~~~~~~~~^^^^^^
  File "/usr/lib/python3.13/site-packages/ramalama/model.py", line 101, in run
    exec_cmd(exec_args)
    ~~~~~~~~^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/ramalama/common.py", line 41, in exec_cmd
    return os.execvp(args[0], args)
           ~~~~~~~~~^^^^^^^^^^^^^^^
  File "<frozen os>", line 611, in execvp
  File "<frozen os>", line 653, in _execvpe
  File "<frozen os>", line 644, in _execvpe
FileNotFoundError: [Errno 2] No such file or directory

This is using an install with my modified spec file which is in my test copr

(I first hit this trying to run ramalama inside a toolbox container, but also get it in a vm with --nocontainer.)

juhp commented 1 week ago

Or is this because I don't have ollama (etc) installed locally say?

ericcurtin commented 1 week ago

Looking at roughly model.py line 101. It's because you don't have llama.cpp installed on your system, looking at the code, it's trying to exec llama-server or llama-cli and you don't have it on your system:

Users who don't use containers are gonna run into this kinda thing all the time. These AI frameworks are complex, --nocontainer is for advanced Linux users and macOS users.

Note if you want to use with a GPU, you will likely have to build llama.cpp in a very specific way.

Something that could be a nice contribution is to make this error message clearer, so if someone else tries this in the future they have a more informative message (they can see it's specifically llama-server/llama-cli that it is trying to execute and failing on).

@juhp we don't use Ollama directly, we have our own implementation of an Ollama client written from scratch.