Installation on MacBook Pro M3 Notebook with errors (Modules not in file in requirement.txt AND Failed to import from vllm._C!)

[hklein42]

Hi,

Installations errors:

Modules not in file in requirement.txt: 1023 pip install byaldi 1025 pip install docx2pdf 1027 pip install vllm 1029 pip install google.generativeai 1031 pip install groq 1033 pip install markdown 1035 pip install vllm._C

Log: localgpt-vision) hklein42@mcbox localGPT-Vision % python app.py
WARNING 10-09 18:02:00 _custom_ops.py:18] Failed to import from vllm._C with ModuleNotFoundError("No module named 'vllm._C'") INFO 10-09 18:02:00 importing.py:10] Triton not installed; certain GPU-related functions will not be available. 2024-10-09 18:02:01,351 - INFO - main - Application started.

• Serving Flask app 'app'
• Debug mode: on WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
• Running on http://127.0.0.1:5050 Press CTRL+C to quit
• Restarting with stat WARNING 10-09 18:02:02 _custom_ops.py:18] Failed to import from vllm._C with ModuleNotFoundError("No module named 'vllm._C'") INFO 10-09 18:02:02 importing.py:10] Triton not installed; certain GPU-related functions will not be available. 2024-10-09 18:02:03,003 - INFO - main - Application started.
• Debugger is active!
• Debugger PIN: 923-579-722

BG Heinzpeter

[PromtEngineer]

@hklein42 can you help me understand the first part. The packages mentioned in the issue are in the requirements.txt . Are you running into issues while installing from requirements.txt?

vLLM is not supported on macOs but shouldn't be an issue. On MacOs you will not be able to use Pixtral, and Molmo because of their dependcy on certain packages that are not supported. But you should be able to run llama and qwen.

[deadsunrise]

@hklein42 can you provide more info, it's working fine for me on a Macstudio M2 Max 192GB on the latest sequoia dev beta

[sputnick]

Same issue on an M1 but it fails most likely because vllm>=0.6.1.post1 stops pip with an error and then you need to manually install the rest. Moving it to the end of the file didn't solve the issue.

[hklein42]

ThxVielen Dank im Voraus Mit freundlichen Grüßen / with kind regardsHeinzpeter KleinMobil :  +49 (151) 23056041mail @. E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. Das unerlaubte  Kopieren sowie die unbefugte Weitergabe dieser E-Mail sind nicht gestattet.Am 10.10.2024 um 15:03 schrieb Eirikur S. Hrafnsson @.>: Same issue on an M1 but it fails most likely because vllm>=0.6.1.post1 stops pip with an error and then you need to manually install the rest. Moving it to the end of the file didn't solve the issue.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>