search

Infini-AI-Lab / Sequoia

scalable and robust tree-based speculative decoding algorithm
282 stars 29 forks source link

Rotary fix #7

Open poedator opened 3 months ago

poedator commented 3 months ago

current code is not compatible with transformers 4.39 + because of changed rotary functions. Fix: copied these functions from transformers==4.37.2