Make gradient clipping a component

Modalities / modalities

Modalities, a PyTorch-native framework for distributed and reproducible foundation model training.

MIT License

61 stars 5 forks source link

Make gradient clipping a component #85

Closed le1nux closed 5 months ago

le1nux commented 7 months ago

Currently, the gradient clipping is instantiated within the main.py and then passed to the Trainer component. Once we have the Trainer as a component, that is part of the hierarchical instantiation (see issue #83), we can add gradient clipping as a dependency of the Trainer component.

see PR #80 for the current state.

le1nux commented 5 months ago

integrated by PR #120