search

triton-inference-server / fastertransformer_backend

BSD 3-Clause "New" or "Revised" License
412 stars 133 forks source link

Feature request: Conversion from GPTBigCodeForCausalLM / Starcoder #132

Open michaelfeil opened 1 year ago

michaelfeil commented 1 year ago

Is it possible to integrate converter scripts for the GPTBigCodeForCausalLM architecture from the transformers libary?

This would enable integration of models like Starcoder / Santacoder.

With this, community projects like https://github.com/fauxpilot/fauxpilot/issues/200 would have a great benefit.

Xingxiangrui commented 1 year ago

Agree ! Maybe starcoder is the SOTA model until 2023.05.25 of the <20B models. It would bring great benefits to the people who use it.