Pascal family cards support.

bigscience-workshop / petals

🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading

https://petals.dev

MIT License

9.11k stars 512 forks source link

Pascal family cards support. #605

Open akamaus opened 2 months ago

akamaus commented 2 months ago

It's said in the FAQ that models are served in FP16 to conserve the GPU memory. What's about support for cards like 1080Ti than? AFAIK they have terrible performance for FP16 (something like 150GFLOP/S compared to 10TFLOP/S on FP32)