This PR introduces several enhancements and refactoring changes, particularly focusing on sequence-to-sequence tasks and model training support.
Features:
Added seq2seq distillation support to leverage knowledge distillation for sequence-to-sequence models (1eebdcb).
Support for T5 models training has been implemented, extending the library's versatility for different model types (7957fb3).
Added language model and summarization scorers to provide a range of evaluation metrics (b6f590f).
Introduced the Seq2SeqTransformerDataModule to handle data related operations for sequence-to-sequence tasks (3f82a99).
Refactoring:
The code has been restructured to improve readability and formatting, making it more maintainable and easier to understand (a41d871).
Updated the perplexity calculation and logging to reflect more accurate measurements (7862746).
Refactored the distiller classes to accommodate the sequence classification task more effectively (7da19ed).
Improved base classes for consistency and enhancement in handling sequence classification and seq2seq tasks (3e4c7b0, 56bc51e).
Restructured scorers for better organization and usage within the library (75a903d).
Transitioned to using the BaseSequenceClassificationTransformerModule to streamline the underlying model handling (d54fa12).
Separated base classes for Transformer modules to establish a clearer hierarchy and improve the extension capabilities of the library (94c9b4d).
Other Changes:
Added evaluate as a dependency to assist in the model evaluation process (bc7d95e).
Enhanced the BaseDataModule with detailed docstrings, improving the documentation and understandability of the data handling process (6ad0d0d).
These updates aim to solidify bert-squeeze's position as a versatile tool for training and deploying sequence-to-sequence models while ensuring high code quality and performance
This PR introduces several enhancements and refactoring changes, particularly focusing on sequence-to-sequence tasks and model training support.
Features:
Refactoring:
Other Changes:
Enhanced the BaseDataModule with detailed docstrings, improving the documentation and understandability of the data handling process (6ad0d0d).
These updates aim to solidify
bert-squeeze's position as a versatile tool for training and deploying sequence-to-sequence models while ensuring high code quality and performance