Optional dependency on transformers

[DuarteMRAlves]

🚀 The feature, motivation and pitch

I want to utilize the liger-kernel fused operations on a codebase but do not need the requirement for transformers. However, when I import the liger_kernel.transformers package, it also imports the transformers library.

Alternatives

I can install transformers as a dependency, but it should not be required if it is not used.

Additional context

I understand that one of the goals of this project is the seamless drop in replacement for transformer models, but I believe it should be possible to import the operations without such a requirement.

[yundai424]

if i understand correctly the only place that pulls transformers in when trying to import liger_kernel.transformers is here. It's possible to make it an optional import (i.e. only import auto_model when transformers is available). Wdyt? @shimizust @ByronHsu

[shimizust]

Good catch, we should wrap that in optional import. Will take this

[shimizust]

@yundai424 I think the monkeypatch functions are also pulling in transformers via the imports at top as well (e.g. from liger_kernel.transformers.model.mistral import lce_forward as mistral_lce_forward). Will move these down into the individual functions

[shimizust]

Thinking about it more, @DuarteMRAlves what exactly are you trying to import? The raw ops can be imported from liger_kernel.ops and don't require transformers. Everything under liger_kernel.transformers is meant for replacing modules in specific models added in the transformers library. So wondering what you're using here if also not using transformers