search

IntelLabs / Hardware-Aware-Automated-Machine-Learning

MIT License
23 stars 4 forks source link

mismatch when inference #12

Open ziyanxzy opened 5 months ago

ziyanxzy commented 5 months ago

when i run the example for this repo: image

it will have the issue: image

jpablomch commented 5 months ago

Thanks, @ziyanxzy, for your interest in Shears! Please check if your environment matches the setup described here. If you intend only to perform inference, apply the PEFT inference patch to PEFT.

git clone https://github.com/huggingface/peft.git
cd peft && git checkout v0.5.0 && git apply --ignore-space-change --ignore-whitespace peft-modifications-for-shears-inference-usage.patch&& pip install -e . && cd ..

Ref: https://huggingface.co/IntelLabs/shears-mpt-7b-50-gsm8k-heuristic-adapter#how-to-use.