Add tests for distributed training

[aniketmaurya]

Is your feature request related to a problem? Please describe.

DDP is supported by HF Accelerate but it is not in the test. Also, there is no example for a demo of this feature.

Describe the solution you'd like

Add code example and test case.

Describe alternatives you've considered

A clear and concise description of any alternative solutions or features you've considered.

Additional context

Add any other context or screenshots about the feature request here.

[github-actions[bot]]

Stale issue message