search

Preemo-Inc / text-generation-inference

Apache License 2.0
200 stars 22 forks source link

refactor(server): Add `AutoCausalLM` to avoid skipping `CausalLM`'s constructor #5

Closed Atry closed 1 year ago

Atry commented 1 year ago

Currently BLOOMSharded is a subclass of CausalLM, while it skips CausalLM's constructor. This is a surprising behavior that we might want to avoid.

This PR extracts CausalLM's constructor to AutoCausalLM to detect settings from model_id, so that we don't have to skip CausalLM's constructor in custom models.

michaelfeil commented 1 year ago

Makes sense to me. On thought HF transformers has high redundancy in their code, keeping inheritance footprint rather minimal. https://github.com/huggingface/transformers/tree/main/src/transformers/models This enabled running and integrating the latest changes fast.

Atry commented 1 year ago

Changes in this PR is already in our internal repository and get tested there. The possibility to break this repository is minimal but not zero because of the lack of publicly visible CI. I am merging this PR. If you encountered any issue, feel free to report it.

Atry commented 1 year ago

Exactly, I think modularization is a goal that we want to do different from HuggingFace's repositories.