exo labs seems to be suffering form a possible memory leak

exo-explore / exo

Run your own AI cluster at home with everyday devices 📱💻 🖥️⌚

GNU General Public License v3.0

10.83k stars 618 forks source link

exo labs seems to be suffering form a possible memory leak #366

Open amirvenus opened 1 week ago

amirvenus commented 1 week ago

Hi,

When I run the exo command on mac and start the inferences using the completion REST API endpoint, the Python process seems to be increasingly using more and more memory.

I have put a delay of 10 seconds between each request but eventually the system crashes due to running out of memory.

Using the latelt macOS on M3 Max amd M2 Ultra

amirvenus commented 1 week ago

/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown warnings.warn('resource_tracker: There appear to be %d '