Regarding the issue of mlx_lm. tokenizer_utils

[fangxuezheng]

I repeatedly confirmed that the mlx and mlx_lm modules have been installed, but tinychat reported an error when adjusting the model during the conversation. The specific error is as follows: resp = await request_handler(request) resp = await handler(request) tokenizer = await resolve_tokenizer(shard.model_id) ModuleNotFoundError: No module named 'mlx_lm.tokenizer_utils'

How to solve it？？

[stephanj]

It seems that despite the mlx and mlx_lm modules being installed, there's still an issue with importing mlx_lm.tokenizer_utils. This could be due to a few reasons:

1. Installation Issues: The mlx_lm package might not be installed correctly or completely.
2. Path Issues: Python might not be able to find the mlx_lm package in its search path.
3. Version Mismatch: There might be a version incompatibility between mlx and mlx_lm.

Let's try to resolve this step by step:

1. First, let's verify the installation of mlx and mlx_lm:

   pip list | grep mlx

   This should show both mlx and mlx_lm if they're installed.

2. If they're not listed, or to ensure you have the latest versions, try reinstalling:

   pip uninstall mlx mlx_lm -y
   pip install mlx mlx_lm

3. If the issue persists, let's try to locate where Python is looking for these modules:

   import sys
   import mlx
   print(sys.path)
   print(mlx.__file__)

   Run this in a Python interpreter. It will show the directories Python searches for modules and where mlx is located.

4. If mlx_lm is not in a directory listed in sys.path, you might need to add its location to your PYTHONPATH:

   export PYTHONPATH=$PYTHONPATH:/path/to/mlx_lm

   Replace /path/to/mlx_lm with the actual path.

5. If you're still having issues, let's try to manually import the module and see where it fails:

   import mlx_lm
   from mlx_lm import tokenizer_utils

   This might give us more detailed error information.

6. As a workaround, if you can't get mlx_lm.tokenizer_utils to import, you might need to modify the code to use a different tokenizer. For example, you could use the Hugging Face tokenizers:

   from transformers import AutoTokenizer
   
   async def resolve_tokenizer(model_id: str):
      return AutoTokenizer.from_pretrained(model_id)

   This would require installing the transformers library:

   pip install transformers

7. If you're on Windows, ensure you're using a version of mlx and mlx_lm that's compatible with Windows, as these libraries were originally designed for Apple Silicon.

8. Check if there are any specific installation instructions or known issues for mlx_lm on its GitHub page or documentation.

If none of these steps resolve the issue, please provide:

1. The output of pip list | grep mlx
2. The full traceback of the error
3. The content of the resolve_tokenizer function

With this information, we can better diagnose and resolve the problem.

[fangxuezheng]

The above is a screenshot of the relevant operation, mlx confirms that it has been installed. The environment variables have also been set, but the module file cannot be found, and it still reports the same error. Could you please take a look at how I can solve it next? Thank you

[stephanj]

You should upgrade to the latest mlx, I have the following working versions:

$ pip list | grep mlx
mlx                          0.16.0
mlx-lm                    0.16.1

[fangxuezheng]

ok ,conda install -c conda-forge mlx-lm

[AlexCheema]

Is this resolved?

If not, can you paste the entire error message above the one you already sent. There should be more.

[JKYtydt]

@AlexCheema 我也遇到了这个问题，但是我无法安装最新版本的mlx和mlx_lm

如果执行pip install mlx mlx_lm，则会默认安装Successfully installed mlx-0.2.0 mlx_lm-0.0.13
若执行pip install mlx==0.16.0  mlx_lm==0.16.1，则会有如下报错
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
ERROR: Could not find a version that satisfies the requirement mlx==0.16.0 (from versions: 0.2.0)
ERROR: No matching distribution found for mlx==0.16.0