search

anarchy-ai / LLM-VM

irresponsible innovation. Try now at https://chat.dev/
https://anarchy.ai/
MIT License
465 stars 150 forks source link

feat: nix package management #428

Closed collinarnett closed 1 week ago

collinarnett commented 6 months ago

This PR adds Nix as a method of installing and running LLM-VM. With this PR, as long as the flake experimental feature is enabled with Nix, users can easily run the LLM-VM server with:

$ nix run github:collinarnett/LLM-VM/nix

Users can also run the server with cuda support baked in:

$ nix run --impure github:collinarnett/LLM-VM/nix#llm_vm_server-cuda

(the impure is needed here in order for nixGL to find the host nvidia drivers)

Developers also benefit from this PR with the ability to start a shell with a python environment including LLM-VM.

$ nix develop github:collinarnett/LLM-VM/nix                    

[collin@vampire:~]$ python
Python 3.11.6 (main, Oct  2 2023, 13:45:54) [GCC 12.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from llm_vm.client import Client

From here tests can be run by starting the server with the first command then running the tests in the developer shell.

In total, the following outputs are provided.

$ nix flake show github:collinarnett/LLM-VM/nix
github:collinarnett/LLM-VM/2cb0b872c4cbec7ea655c6bb4ec178d6e69f9bd2
├───apps
│   └───x86_64-linux
│       ├───default: app
│       ├───llm_vm_run_agent: app
│       ├───llm_vm_run_agent-cuda: app
│       ├───llm_vm_run_agent-rocm: app
│       ├───llm_vm_run_agent_backwards_chaining: app
│       ├───llm_vm_run_agent_backwards_chaining-cuda: app
│       ├───llm_vm_run_agent_backwards_chaining-rocm: app
│       ├───llm_vm_run_agent_flat: app
│       ├───llm_vm_run_agent_flat-cuda: app
│       ├───llm_vm_run_agent_flat-rocm: app
│       ├───llm_vm_run_agent_rebel: app
│       ├───llm_vm_run_agent_rebel-cuda: app
│       ├───llm_vm_run_agent_rebel-rocm: app
│       ├───llm_vm_server: app
│       ├───llm_vm_server-cuda: app
│       └───llm_vm_server-rocm: app
├───devShells
│   └───x86_64-linux
│       ├───cuda: development environment 'nix-shell'
│       ├───default: development environment 'interactive-python3-3.11.6-environment'
│       └───rocm: development environment 'nix-shell'
└───packages
    └───x86_64-linux
        ├───cuda: package 'python3.11-llm_vm-0.1.55'
        ├───default: package 'python3.11-llm_vm-0.1.55'
        └───rocm: package 'python3.11-llm_vm-0.1.55'

I would be grateful if others could review this PR by installing nix via the The Determinate Nix Installer and running the commands I included above. I do not have a rocm enabled video card so I would appreciate for those that do to leave their feedback.

Also let me know your general experience. There may be things I can do to make things better.

collinarnett commented 6 months ago

One more thing. Some of the executables installed from LLM-VM don't work due to import errors. 

  File "/nix/store/02w0r9mqq5z19gmzxf7qgcp8m4cnlypc-python3.11-llm_vm-0.1.55/lib/python3.11/site-packages/llm_vm/agents/FLAT/models/utils/tool_picker_model/tool_picker_model_data.py", line 25, in __get_toolpicker_model
    json_data: List[QuestionSplitModelJSONData] = json.load(open(file_path, "r"))["data"]
                                                            ^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/nix/store/02w0r9mqq5z19gmzxf7qgcp8m4cnlypc-python3.11-llm_vm-0.1.55/lib/python3.11/site-packages/llm_vm/agents/FLAT/models/raw_data/toolpicker_data.json'

I imagine this issue is limited to the agents since they are run relative to the project root in the README.

collinarnett commented 1 week ago

Don't want to work on this anymore and this repo seems dead.