Qwen Segmentation Fault after logN PR merged.

intel / xFasterTransformer

Apache License 2.0

374 stars 65 forks source link

Closed Duyi-Wang closed 8 months ago

Duyi-Wang commented 8 months ago

This issue occurs after this PR https://github.com/intel/xFasterTransformer/pull/215 merged. Reproduce cmd: bash ./run_benchmark.sh -m qwen-7b -d bf16 -s 1 -bs 1 -in 4096 -out 2048 -i 1

a3213105 commented 8 months ago

please using qwen_convert.py to re-convert the qwen model. new parameter "seq_length" add for logN and NTK rotary.

I also will commit a PR to check the parameter and rise a warning if "seq_length" is unset.

Duyi-Wang commented 8 months ago

Fixed by add seq_length into config.ini