[BOUNTY - $200] Windows native support

AlexCheema commented 2 months ago

Windows is largely untested. Don't know of anyone who has it working natively, only in WSL
Known issues on windows: https://github.com/exo-explore/exo/issues/184 , tinygrad not officially supported
Perhaps llama.cpp backend #167 will enable working inference on windows if tinygrad doesn't work or PyTorch (working on #139)

omarei-omoto commented 2 months ago

PLEAAASE THIS

Shivp1413 commented 1 month ago

I made changes in main.py file, and now its working for windows as well. But I get following error: Error: Failed to fetch completions: Error processing prompt (see logs with DEBUG>=2): No module named '_posixshmem' main.zip

mercurymason commented 1 month ago

The '_posixshmem' module is associated with POSIX shared memory, which afaik is not available on Windows. This error due to tinygrad's reliance on Unix-specific features. It doesn't seem like this problem will be soon resolved by tinygrad, as they have discontinued official Windows support in tinygrad version 0.7.0.

WSL would likely enable tinygrad compatibility on Windows, but kind of breaks the native in "native support". As @AlexCheema mentioned, llama.ccp or pytorch backend may provide a fix -- also, JAX is compatible with Windows.

exo-explore / exo

[BOUNTY - $200] Windows native support #186