search

exo-explore / exo

Run your own AI cluster at home with everyday devices 📱💻 🖥️⌚
GNU General Public License v3.0
7.06k stars 375 forks source link

Issue with loading weights #231

Open JohnLee1360 opened 1 day ago

JohnLee1360 commented 1 day ago

I am trying to use two MacBooks to run a Llama 8B model, but I can't load weights from the model to inference and stuck in 0 progress...

Screenshot 2024-09-24 at 12 03 28

Here's info of my equipments: node1: MacBook Air 16GB with M3 chip node2: MacBook Pro 16GB with M1 chip(Intel based)

Since my machine resource is limited, all of my machines run in tinygrad inference engine rather than in MLX. And I also wonder why my MacBook Pro shows 0TFLOPS?

I will be so appreciative if someone can offer me help~

AlexCheema commented 20 hours ago

Can you try running with SUPPORT_BF16=0

JohnLee1360 commented 4 hours ago

Thanks for the reply! I assume that SUPPORT_BF16=0 means smaller quantized weights accuracy, right? I gave it a shot, but it doesn't work. Maybe the problem stems from the different architecture of chips?