Error replicating section of blog "Personal Copilot"

[zequeira]

Hi,

I'm trying to reproduce the section "How do I run it locally?" from this blog post:

Personal Copilot: Train Your Own Coding Assistant (huggingface.co)

When I execute step number 2, I get the following error:

(copilot) --@-- python % pip install -e "."
Obtaining file:///Users/--/GitLab/---copilot/mlc-llm/python
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [15 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/Users/--/GitLab/---copilot/mlc-llm/python/setup.py", line 47, in <module>
          LIB_LIST, __version__ = get_lib_path()
                                  ^^^^^^^^^^^^^^
        File "/Users/--/GitLab/---copilot/mlc-llm/python/setup.py", line 27, in get_lib_path
          libinfo["find_lib_path"]("mlc_llm")[0],
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/Users/--/GitLab/---copilot/mlc-llm/python/./mlc_chat/libinfo.py", line 72, in find_lib_path
          raise RuntimeError(message)
      RuntimeError: Cannot find libraries: libmlc_llm.dylib
      List of candidates:
      /Users/--/GitLab/---copilot/mlc-llm/python/mlc_chat/libmlc_llm.dylib
      /Users/-/opt/miniconda3/envs/copilot/lib/libmlc_llm.dylib
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

However, if I do:

cd ..

and execute:

pip install -e "."

from:

/Users/--/GitLab/---copilot/mlc-llm

The install is successful.

Then, Step number 3 (Model compilation) is successful.

But, Step 5, fails. And I get the following error:

/Users/--/opt/miniconda3/envs/copilot/bin/python: Error while finding module specification for 'mlc_chat.rest' (ModuleNotFoundError: No module named 'mlc_chat')

However, if I install:

pip install --pre --force-reinstall mlc-chat-nightly -f https://mlc.ai/wheels

and execute Step 5, then I get the following error:

(copilot) --@-- mlc-llm % python -m mlc_chat.rest --model dist/starcoder-personal-copilot----q4f16_1/params --lib-path dist/starcoder-personal-copilot----q4f16_1/starcoder-personal-copilot----q4f16_1-metal.so
INFO:     Started server process [9607]
INFO:     Waiting for application startup.
ERROR:    Traceback (most recent call last):
  File "/Users/--/opt/miniconda3/envs/copilot/lib/python3.11/site-packages/starlette/routing.py", line 677, in lifespan
    async with self.lifespan_context(app) as maybe_state:
  File "/Users/--/opt/miniconda3/envs/copilot/lib/python3.11/contextlib.py", line 204, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/--/opt/miniconda3/envs/copilot/lib/python3.11/site-packages/mlc_chat/rest.py", line 137, in lifespan
    chat_mod = ChatModule(
               ^^^^^^^^^^^
  File "/Users/--/opt/miniconda3/envs/copilot/lib/python3.11/site-packages/mlc_chat/chat_module.py", line 710, in __init__
    self._reload(self.model_lib_path, self.model_path, user_chat_config_json_str)
  File "/Users/--/opt/miniconda3/envs/copilot/lib/python3.11/site-packages/mlc_chat/chat_module.py", line 922, in _reload
    self._reload_func(lib, model_path, app_config_json)
  File "tvm/_ffi/_cython/./packed_func.pxi", line 332, in tvm._ffi._cy3.core.PackedFuncBase.__call__
  File "tvm/_ffi/_cython/./packed_func.pxi", line 263, in tvm._ffi._cy3.core.FuncCall
  File "tvm/_ffi/_cython/./packed_func.pxi", line 252, in tvm._ffi._cy3.core.FuncCall3
  File "tvm/_ffi/_cython/./base.pxi", line 182, in tvm._ffi._cy3.core.CHECK_CALL
  File "/Users/--/opt/miniconda3/envs/copilot/lib/python3.11/site-packages/tvm/_ffi/base.py", line 481, in raise_last_ffi_error
    raise py_err
tvm.error.InternalError: Traceback (most recent call last):
  File "/Users/catalyst/Workspace/mlc-ai-package-self-runner/_work/package/package/tvm/src/runtime/dso_library.cc", line 125
InternalError: Check failed: (lib_handle_ != nullptr) is false: Failed to load dynamic shared library dist/starcoder-personal-copilot----q4f16_1/starcoder-personal-copilot----q4f16_1-metal.so dlopen(dist/starcoder-personal-copilot----q4f16_1/starcoder-personal-copilot----q4f16_1-metal.so, 0x0005): tried: 'dist/starcoder-personal-copilot----q4f16_1/starcoder-personal-copilot----q4f16_1-metal.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OSdist/starcoder-personal-copilot----q4f16_1/starcoder-personal-copilot----q4f16_1-metal.so' (no such file), '/Users/--/opt/miniconda3/envs/copilot/bin/../lib/dist/starcoder-personal-copilot----q4f16_1/starcoder-personal-copilot----q4f16_1-metal.so' (no such file), '/usr/lib/dist/starcoder-personal-copilot----q4f16_1/starcoder-personal-copilot----q4f16_1-metal.so' (no such file, not in dyld cache), 'dist/starcoder-personal-copilot----q4f16_1/starcoder-personal-copilot----q4f16_1-metal.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/Users/--/GitLab/---copilot/mlc-llm/dist/starcoder-personal-copilot----q4f16_1/starcoder-personal-copilot----q4f16_1-metal.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/Users/--/GitLab/---copilot/mlc-llm/dist/starcoder-personal-copilot----q4f16_1/starcoder-personal-copilot----q4f16_1-metal.so' (no such file), '/Users/---/GitLab/---copilot/mlc-llm/dist/starcoder-personal-copilot----q4f16_1/starcoder-personal-copilot----q4f16_1-metal.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64'))

ERROR:    Application startup failed. Exiting.

Maybe @pacman100 could help with this? Any help would be appreciated. Best.

[pacman100]

Hello, I will update the section with new instructions as per the latest changes in the mlc-llm repo. This might take some time though.