search

pytorch / torchchat

Run PyTorch LLMs locally on servers, desktop and mobile
BSD 3-Clause "New" or "Revised" License
3.13k stars 195 forks source link

torchchat generate requires network connection, even if models are cached #1014

Open malfet opened 1 month ago

malfet commented 1 month ago

🐛 Describe the bug

Run python3 torchchat.py generate stories110M on a system with a bad network connection will hang for 90+ sec before it starts generatin anything

Versions

Not sure why pytorch collect_env is of any use here...

malfet commented 1 month ago

This happens while one import torchao.quantization.quant_api