search

NetEase-FuXi / EETQ

Easy and Efficient Quantization for Transformers
Apache License 2.0
180 stars 14 forks source link

Bug: Import of shard_checkpoint from transformers fails #34

Open BenjaminBossan opened 5 days ago

BenjaminBossan commented 5 days ago

This import is failing with the latest transformers version:

https://github.com/NetEase-FuXi/EETQ/blob/81e0b14d64088d58ef6acd2c8f3e788d59324407/python/eetq/models/base.py#L13

I checked how others solved this and it looks like in AWQ, this import could be removed by using the save_torch_state_dict from huggingface_hub insted.

https://github.com/casper-hansen/AutoAWQ/compare/v0.2.6...v0.2.7#diff-068b6780bfd41edd049c82b641c984b3f7e5278b8ef412d29cdba6a62663704aR305-R310

dtlzhuangz commented 4 days ago

We will fix the bug soon after we finish our current work.