VITA-Group Ms-PoE issues

VITA-Group / Ms-PoE

"Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding" Zhenyu Zhang, Runjin Chen, Shiwei Liu, Zhewei Yao, Olatunji Ruwase, Beidi Chen, Xiaoxia Wu, Zhangyang Wang.

MIT License

21 stars 2 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

The error in the implementiation of the MsPoELlamaRotaryEmbedding

#5 HaozheZhao opened 2 months ago
0
Problems in _head_wise_statistics() function

#4 JingfenQiao opened 4 months ago
1
Support more models and flash attention 2

#3 yuyijiong opened 5 months ago
1
Some question about inference.py on ZeroScrolls

#2 altctrl00 opened 7 months ago
0
some questions when generating

#1 Michelleable closed 7 months ago
0

VITA-Group / Ms-PoE

issues

The error in the implementiation of the MsPoELlamaRotaryEmbedding

Problems in _head_wise_statistics() function

Support more models and flash attention 2

Some question about inference.py on ZeroScrolls

some questions when generating