search

meta-llama / llama-stack

Model components of the Llama Stack APIs
MIT License
3.42k stars 489 forks source link

llama cli does not work on mac Sequoia #143

Open kavukcutolga opened 1 week ago

kavukcutolga commented 1 week ago

Hi, I have installed llama cli, but getting below error for any command. I installed it with pip3 install llama-stack. This prevents further usage of llama-stack, could you please guide to address this issue?

 llama stack build --list-templates

Traceback (most recent call last):
  File "/usr/local/bin/llama", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/local/lib/python3.11/site-packages/llama_stack/cli/llama.py", line 42, in main
    parser = LlamaCLIParser()
             ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/llama_stack/cli/llama.py", line 32, in __init__
    StackParser.create(subparsers)
  File "/usr/local/lib/python3.11/site-packages/llama_stack/cli/subcommand.py", line 16, in create
    return cls(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/llama_stack/cli/stack/stack.py", line 33, in __init__
    StackListProviders.create(subparsers)
  File "/usr/local/lib/python3.11/site-packages/llama_stack/cli/subcommand.py", line 16, in create
    return cls(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/llama_stack/cli/stack/list_providers.py", line 21, in __init__
    self._add_arguments()
  File "/usr/local/lib/python3.11/site-packages/llama_stack/cli/stack/list_providers.py", line 25, in _add_arguments
    from llama_stack.distribution.distribution import stack_apis
  File "/usr/local/lib/python3.11/site-packages/llama_stack/distribution/distribution.py", line 13, in <module>
    from llama_stack.apis.agents import Agents
  File "/usr/local/lib/python3.11/site-packages/llama_stack/apis/agents/__init__.py", line 7, in <module>
    from .agents import *  # noqa: F401 F403
    ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/llama_stack/apis/agents/agents.py", line 155, in <module>
    class MemoryToolDefinition(ToolDefinitionCommon):
  File "pydantic/main.py", line 197, in pydantic.main.ModelMetaclass.__new__
  File "pydantic/fields.py", line 497, in pydantic.fields.ModelField.infer
  File "pydantic/fields.py", line 476, in pydantic.fields.ModelField._get_field_info
ValueError: cannot specify `Annotated` and value `Field`s together for 'query_generator_config'
codeseeker-dhruv commented 1 week ago

The specific issue comes from the use of Annotated and Field together in a Pydantic model. This issue might stem from changes introduced in Pydantic v2.x, while the llama-stack may have been developed for Pydantic v1.x.

kavukcutolga commented 1 week ago

Any ideas on how to workaround this ?

codeseeker-dhruv commented 1 week ago

Any ideas on how to workaround this ?

By downgrading Pydantic to version v1.x and use a virtual environment to isolate and manage dependencies safely also you can patch the code if the issue persists and downgrading is not feasible.

clearstorm-tech commented 6 days ago

This has just appeared for me and downgrading is not working

ashwinb commented 6 days ago

Sorry for the delay in responding here. @kavukcutolga @clearstorm-tech -- you need to upgrade Pydantic to version >= 2. Can you see what version of pydantic you have in the environment where you are trying to use llama stack CLI?

We will make sure we add versioning requirements properly in our dependencies.

clearstorm-tech commented 6 days ago

@ashwinb Thank you for your response.

  pydantic  2.9.2   
  pydantic-core  2.23.4

This is the same before and after pulling the update from main.

However I can run the actual commands from the table no problem. 

llama stack build --template local-ollama --name stack
ashwinb commented 6 days ago

I think there's something else going on. Can you please run a couple commands for me:

Also -- if you want, could you hop on over to our Discord at https://discord.com/channels/1257833999603335178/1257834000190275586 and we can debug this more synchronously perhaps.

ashwinb commented 6 days ago

By the way, we also updated packages to 0.0.37 now -- can you re-install and give it a try again?

clearstorm-tech commented 6 days ago

@ashwinb

I see what is going in now. When I installed I have a stack env, but when I build I get a llamastack-stack env. I guess I have not switched to the llama-stack env. 

  # conda environments:
  #
  base                    ~/miniconda3
  llamastack-stack        ~/miniconda3/envs/llamastack-stack
  stack                 *  ~/miniconda3/envs/stack

  (stack) ➜  llama-stack git:(main) ✗ conda activate llamastack-stack && pip list | grep pydantic
  pydantic                                 2.9.2
  pydantic_core                            2.23.4
  (llamastack-stack) ➜  llama-stack git:(main) ✗ llama stack build  --list-templates

The --list-templates command works now I have switched to llamastack-stack env

I guess there was some confusion with the install instructions and then it creates another env on build?

btw - link did not work for me, perhaps expired.

ashwinb commented 6 days ago

Hm, that is still odd actually. Things should have just worked regardless of which environment you had :(

If you switch back to the "stack" environment, do they fail? If so, could you put the output of the entire pip list somewhere?

clearstorm-tech commented 6 days ago

Yes it still fails. Here is a conda list for the failing stack env - created with conda create -n stack python=3.10 from the install instructions

  # packages in environment at ~/miniconda3/envs/stack:
  #
  # Name                    Version                   Build  Channel
  annotated-types           0.7.0                    pypi_0    pypi
  anyio                     4.6.0                    pypi_0    pypi
  blobfile                  3.0.0                    pypi_0    pypi
  bzip2                     1.0.8                h6c40b1e_6
  ca-certificates           2024.9.24            hecd8cb5_0
  certifi                   2024.8.30                pypi_0    pypi
  charset-normalizer        3.3.2                    pypi_0    pypi
  exceptiongroup            1.2.2                    pypi_0    pypi
  filelock                  3.16.1                   pypi_0    pypi
  fire                      0.7.0                    pypi_0    pypi
  fsspec                    2024.9.0                 pypi_0    pypi
  h11                       0.14.0                   pypi_0    pypi
  httpcore                  1.0.6                    pypi_0    pypi
  httpx                     0.27.2                   pypi_0    pypi
  huggingface-hub           0.25.1                   pypi_0    pypi
  idna                      3.10                     pypi_0    pypi
  jinja2                    3.1.4                    pypi_0    pypi
  libffi                    3.4.4                hecd8cb5_1
  llama-models              0.0.36                   pypi_0    pypi
  llama-stack               0.0.36                   pypi_0    pypi
  lxml                      5.3.0                    pypi_0    pypi
  markdown-it-py            3.0.0                    pypi_0    pypi
  markupsafe                2.1.5                    pypi_0    pypi
  mdurl                     0.1.2                    pypi_0    pypi
  ncurses                   6.4                  hcec6c5f_0
  openssl                   3.0.15               h46256e1_0
  packaging                 24.1                     pypi_0    pypi
  pillow                    10.4.0                   pypi_0    pypi
  pip                       24.2            py310hecd8cb5_0
  prompt-toolkit            3.0.48                   pypi_0    pypi
  pycryptodomex             3.21.0                   pypi_0    pypi
  pydantic                  2.9.2                    pypi_0    pypi
  pydantic-core             2.23.4                   pypi_0    pypi
  pygments                  2.18.0                   pypi_0    pypi
  python                    3.10.14              h5ee71fb_1
  python-dotenv             1.0.1                    pypi_0    pypi
  pyyaml                    6.0.2                    pypi_0    pypi
  readline                  8.2                  hca72f7f_0
  regex                     2024.9.11                pypi_0    pypi
  requests                  2.32.3                   pypi_0    pypi
  rich                      13.9.1                   pypi_0    pypi
  setuptools                75.1.0          py310hecd8cb5_0
  sniffio                   1.3.1                    pypi_0    pypi
  sqlite                    3.45.3               h6c40b1e_0
  termcolor                 2.4.0                    pypi_0    pypi
  tiktoken                  0.7.0                    pypi_0    pypi
  tk                        8.6.14               h4d00af3_0
  tqdm                      4.66.5                   pypi_0    pypi
  typing-extensions         4.12.2                   pypi_0    pypi
  tzdata                    2024a                h04d1e81_0
  urllib3                   2.2.3                    pypi_0    pypi
  wcwidth                   0.2.13                   pypi_0    pypi
  wheel                     0.44.0          py310hecd8cb5_0
  xz                        5.4.6                h6c40b1e_1
  zlib                      1.2.13               h4b97444_1
clearstorm-tech commented 3 days ago

To follow on from this. I have found the following. If I install via pip install it works.

I hope this helps, but I shall continue with pip install.

This does not work: conda list above.

 git clone git@github.com:meta-llama/llama-stack.git

  conda create -n stack python=3.10
  conda activate stack

  cd llama-stack <- I get an error if I navigate to this folder and install so I don't cd to this folder 
  ERROR -> llama-stack/llama_stack does not appear to be a Python project: neither 'setup.py' nor 'pyproject.toml' found

  $CONDA_PREFIX/bin/pip install -e .

 llama stack build --list-templates

  Traceback (most recent call last):
    File "/path/to/llama/env/bin/llama", line 8, in <module>
      sys.exit(main())
    File "/path/to/llama/tools/llama_stack/cli/llama.py", line 44, in main
      parser.run(args)
    File "/path/to/llama/tools/llama_stack/cli/llama.py", line 38, in run
      args.func(args)
    File "/path/to/llama/tools/llama_stack/cli/stack/build.py", line 188, in _run_stack_build_command
      self._run_template_list_cmd(args)
    File "/path/to/llama/tools/llama_stack/cli/stack/build.py", line 166, in _run_template_list_cmd
      for spec in available_templates_specs():
    File "/path/to/llama/tools/llama_stack/cli/stack/build.py", line 27, in available_templates_specs
      build_config = BuildConfig(**yaml.safe_load(f))
    File "/path/to/python/env/lib/python3.10/site-packages/pydantic/main.py", line 212, in __init__
      validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
  pydantic_core._pydantic_core.ValidationError: 2 validation errors for BuildConfig
  name
    Field required [type=missing, input_value={'built_at': '2024-09-30T...uting_key': 'vector'}]}}, input_type=dict]
      For further information visit https://errors.pydantic.dev/2.9/v/missing
  distribution_spec
    Field required [type=missing, input_value={'built_at': '2024-09-30T...uting_key': 'vector'}]}}, input_type=dict]
      For further information visit https://errors.pydantic.dev/2.9/v/missing

This works:

conda create -n stack python=3.10
conda activate stack
pip install llama-stack
llama stack build --list-templates

conda list 

  # packages in environment at ~/miniconda3/envs/stack:
  #
  # Name                    Version                   Build  Channel
  annotated-types           0.7.0                    pypi_0    pypi
  anyio                     4.6.0                    pypi_0    pypi
  blobfile                  3.0.0                    pypi_0    pypi
  bzip2                     1.0.8                h6c40b1e_6
  ca-certificates           2024.9.24            hecd8cb5_0
  certifi                   2024.8.30                pypi_0    pypi
  charset-normalizer        3.3.2                    pypi_0    pypi
  exceptiongroup            1.2.2                    pypi_0    pypi
  filelock                  3.16.1                   pypi_0    pypi
  fire                      0.7.0                    pypi_0    pypi
  fsspec                    2024.9.0                 pypi_0    pypi
  h11                       0.14.0                   pypi_0    pypi
  httpcore                  1.0.6                    pypi_0    pypi
  httpx                     0.27.2                   pypi_0    pypi
  huggingface-hub           0.25.1                   pypi_0    pypi
  idna                      3.10                     pypi_0    pypi
  jinja2                    3.1.4                    pypi_0    pypi
  libffi                    3.4.4                hecd8cb5_1
  llama-models              0.0.40                   pypi_0    pypi
  llama-stack               0.0.40                   pypi_0    pypi
  lxml                      5.3.0                    pypi_0    pypi
  markdown-it-py            3.0.0                    pypi_0    pypi
  markupsafe                2.1.5                    pypi_0    pypi
  mdurl                     0.1.2                    pypi_0    pypi
  ncurses                   6.4                  hcec6c5f_0
  openssl                   3.0.15               h46256e1_0
  packaging                 24.1                     pypi_0    pypi
  pillow                    10.4.0                   pypi_0    pypi
  pip                       24.2            py310hecd8cb5_0
  prompt-toolkit            3.0.48                   pypi_0    pypi
  pycryptodomex             3.21.0                   pypi_0    pypi
  pydantic                  2.9.2                    pypi_0    pypi
  pydantic-core             2.23.4                   pypi_0    pypi
  pygments                  2.18.0                   pypi_0    pypi
  python                    3.10.15              hce00570_1
  python-dotenv             1.0.1                    pypi_0    pypi
  pyyaml                    6.0.2                    pypi_0    pypi
  readline                  8.2                  hca72f7f_0
  regex                     2024.9.11                pypi_0    pypi
  requests                  2.32.3                   pypi_0    pypi
  rich                      13.9.2                   pypi_0    pypi
  setuptools                75.1.0          py310hecd8cb5_0
  sniffio                   1.3.1                    pypi_0    pypi
  sqlite                    3.45.3               h6c40b1e_0
  termcolor                 2.4.0                    pypi_0    pypi
  tiktoken                  0.8.0                    pypi_0    pypi
  tk                        8.6.14               h4d00af3_0
  tqdm                      4.66.5                   pypi_0    pypi
  typing-extensions         4.12.2                   pypi_0    pypi
  tzdata                    2024a                h04d1e81_0
  urllib3                   2.2.3                    pypi_0    pypi
  wcwidth                   0.2.13                   pypi_0    pypi
  wheel                     0.44.0          py310hecd8cb5_0
  xz                        5.4.6                h6c40b1e_1
  zlib                      1.2.13               h4b97444_1
kavukcutolga commented 2 days ago

I'm having the exact same issue with conda. 

(stack) xx@xx llama-stack % llama stack build  --list-templates             
Traceback (most recent call last):
  File "/Users/xx/miniconda3/envs/stack/bin/llama", line 8, in <module>
    sys.exit(main())
  File "/Users/xx/PycharmProjects/llama-stack/llama_stack/cli/llama.py", line 44, in main
    parser.run(args)
  File "/Users/xx/PycharmProjects/llama-stack/llama_stack/cli/llama.py", line 38, in run
    args.func(args)
  File "/Users/xx/PycharmProjects/llama-stack/llama_stack/cli/stack/build.py", line 188, in _run_stack_build_command
    self._run_template_list_cmd(args)
  File "/Users/xx/PycharmProjects/llama-stack/llama_stack/cli/stack/build.py", line 166, in _run_template_list_cmd
    for spec in available_templates_specs():
  File "/Users/xx/PycharmProjects/llama-stack/llama_stack/cli/stack/build.py", line 27, in available_templates_specs
    build_config = BuildConfig(**yaml.safe_load(f))
  File "/Users/xx/miniconda3/envs/stack/lib/python3.10/site-packages/pydantic/main.py", line 212, in __init__
    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 2 validation errors for BuildConfig
name
  Field required [type=missing, input_value={'built_at': '2024-09-30T...uting_key': 'vector'}]}}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.9/v/missing
distribution_spec
  Field required [type=missing, input_value={'built_at': '2024-09-30T...uting_key': 'vector'}]}}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.9/v/missing