search

bigscience-workshop / petals

🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
https://petals.dev
MIT License
9.26k stars 526 forks source link

NotImplementedError: #599

Closed pass-pass-pass closed 4 months ago

pass-pass-pass commented 4 months ago

NotImplementedError: aten::_local_scalar_dense: attempted to run this operator with Meta tensors, but there was no abstract impl or Meta kernel registered. You may have run into this message while using an operator with PT2 compilation APIs (torch.compile/torch.export); in order to use this operator with those APIs you'll need to add an abstract impl.Please see the following doc for next steps: https://docs.google.com/document/d/1_W62p8WJOQQUzPsJYa7s701JXt0qf2OfLub2sbkHOaU/edit

does anyone else run into the same issue

borzunov commented 4 months ago

Hi @pass-pass-pass, this has been fixed in #599. Don't forget to update to the latest version from the repo:

pip install git+https://github.com/bigscience-workshop/petals