search

IntelLabs / Hardware-Aware-Automated-Machine-Learning

MIT License
14 stars 4 forks source link

mismatch when inference #12

Open ziyanxzy opened 1 week ago

ziyanxzy commented 1 week ago

when i run the example for this repo: image

it will have the issue: image

jpablomch commented 1 week ago

Thanks, @ziyanxzy, for your interest in Shears! Please check if your environment matches the setup described here. If you intend only to perform inference, apply the PEFT inference patch to PEFT.

git clone https://github.com/huggingface/peft.git
cd peft && git checkout v0.5.0 && git apply --ignore-space-change --ignore-whitespace peft-modifications-for-shears-inference-usage.patch&& pip install -e . && cd ..

Ref: https://huggingface.co/IntelLabs/shears-mpt-7b-50-gsm8k-heuristic-adapter#how-to-use.