search

mistralai / mistral-inference

Official inference library for Mistral models
https://mistral.ai/
Apache License 2.0
9.58k stars 845 forks source link

Python 3.11.6 compatibility #45

Open MasterLivens opened 11 months ago

MasterLivens commented 11 months ago

(venv) E:\AI\mistral-7B-v0.1\mistral-src>pip install Fire Requirement already satisfied: Fire in e:\ai\mistral-7b-v0.1\venv\lib\site-packages (0.5.0) Requirement already satisfied: six in e:\ai\mistral-7b-v0.1\venv\lib\site-packages (from Fire) (1.16.0) Requirement already satisfied: termcolor in e:\ai\mistral-7b-v0.1\venv\lib\site-packages (from Fire) (2.3.0)

(venv) E:\AI\mistral-7B-v0.1\mistral-src>python -m main demo E:\AI\mistral-7B-v0.1\model Traceback (most recent call last): File "", line 198, in _run_module_as_main File "", line 88, in _run_code File "E:\AI\mistral-7B-v0.1\mistral-src\main.py", line 140, in fire.Fire({ File "E:\AI\mistral-7B-v0.1\venv\Lib\site-packages\fire\core.py", line 141, in Fire component_trace = _Fire(component, args, parsed_flag_args, context, name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\AI\mistral-7B-v0.1\venv\Lib\site-packages\fire\core.py", line 475, in _Fire component, remaining_args = _CallAndUpdateTrace( ^^^^^^^^^^^^^^^^^^^^ File "E:\AI\mistral-7B-v0.1\venv\Lib\site-packages\fire\core.py", line 691, in _CallAndUpdateTrace component = fn(*varargs, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^ File "E:\AI\mistral-7B-v0.1\mistral-src\main.py", line 124, in demo res, _logprobs = generate( ^^^^^^^^^ File "E:\AI\mistral-7B-v0.1\venv\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "E:\AI\mistral-7B-v0.1\mistral-src\main.py", line 61, in generate prelogits = model.forward( ^^^^^^^^^^^^^^ File "E:\AI\mistral-7B-v0.1\mistral-src\mistral\model.py", line 204, in forward input_metadata = cache.get_input_metadata(seqlens) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\AI\mistral-7B-v0.1\mistral-src\mistral\cache.py", line 192, in get_input_metadata mask = BlockDiagonalCausalMask.from_seqlens(seqlens).make_local_attention(self.sliding_window) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'BlockDiagonalCausalMask' object has no attribute 'make_local_attention'

skyfaker commented 11 months ago

It looks like a problem of xformers, you can try to upgrade it with pip install --upgrade xformers

Alice1998 commented 11 months ago

so what is the xformers version in the final? I tried 0.0.22, 0.0.21, 0.0.16 all facing errors. thanks!

skyfaker commented 11 months ago

so what is the xformers version in the final? I tried 0.0.22, 0.0.21, 0.0.16 all facing errors. thanks!

我的是 xformers==0.0.22.post4，刚刚测试了是可以的