Is there any plan to merge the `modeling_chatglm.py` into the Transformers library?

[iofu728]

Feature request / 功能建议

Hi GLM-4 Team,

Thanks for your great work and the powerful GLM-4 models. I'm currently conducting research on efficient long-context LLMs inference and am trying to implement our method in GLM-4-1M. To reduce the effort, I wanted to ask if there are any plans to merge the modeling_chatglm.py into the Transformers library? This would greatly help our work and benefit the community. Thanks for your help!

Motivation / 动机

To do some experiments on efficient long-context LLMs inference topic.

Your contribution / 您的贡献

I'll open-source the related experiments results and implementation.

[zRzRzRzRzRzRzR]

We also aim to adapt to the Transformers library. However, the current model implementation contains some legacy code that may not fully comply with the requirements of the Transformers library. We will make the necessary modifications as soon as possible and submit a PR to the Transformers library. This may take some time, and we appreciate your understanding.

[iofu728]

Thanks @zRzRzRzRzRzRzR for the response and plan. Looking forward the merged version GLM-4.

[zRzRzRzRzRzRzR]

This issue will be closed first. We will make an announcement after merging it into hf. Thank you for your support.