feature: Adapters finetuning - Githubissues

JulesBelveze / bert-squeeze

🛠️ Tools for Transformers compression using PyTorch Lightning ⚡

https://julesbelveze.github.io/bert-squeeze/

78 stars 10 forks source link

feature: Adapters finetuning #60

Closed JulesBelveze closed 5 months ago

JulesBelveze commented 5 months ago

Implement LtAdapter for adapter-based fine-tuning

This pull request introduces the implementation of the LtAdapter for adapter-based fine-tuning within the bert_squeeze framework.

Changes:

Feature implementation:

Added LtAdapter class for adapter-based training functionality in lt_adapter.py.
Included a new configuration file train_adapter.yaml for setting up LtAdapter training.
Updated train_assistant.py to support the new "adapter" model type.
Integrated LtAdapter into the bert_squeeze.models module for easy access and usage within the framework.

Testing:

Developed a test case in test_train_assistant.py to ensure proper initialization and functioning of TrainAssistant with the LtAdapter model.

Dependencies & Fix:

Modified pyproject.toml to adjust the transformers library version, ensuring compatibility by setting it below version 4.36.0.
Introduced a new dependency on adapters version 0.1.1 to bring additional functionality required for adapter-based models.