Adds simple Accelerate integration example

[tcapelle]

Simple training script on FashionMNIST using accelerate and multiple GPUs inside a notebook.

• Log only on main process.

[review-notebook-app[bot]]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[github-actions[bot]]

The following colabs where changed -colabs/huggingface/Simple_accelerate_integration_wandb.ipynb

[tcapelle]

this one should also be ready

[morganmcg1]

looks good, probably want to have a case to avoid .log being called twice between the last train step and the validation step, so that the step counts line up. We don’t highlight that best practice enough and results in user confusion down the line

[tcapelle]

How would you do that? I am computing metrics on the train DL as I didn't put any validation DL. Look at the run here: https://wandb.ai/capecape/accelerate_fmnist?workspace=user-capecape What one would like is maybe having the epochs line up, but the accuracy at the start is unknown.

[morganmcg1]

@tcapelle you'd need a check to see if it was at a validation step or not. It if is, then you could pass commit=False for the train metrics

[tcapelle]

You mean between these 2:

but it's ugly to pass kwargs to the accelerator:

accelerator.log({"epoch":epoch, "accuracy":accuracy}, log_kwargs={"wandb": {"commit": False}})

[morganmcg1]

ugly, but the best practice way to do it

[tcapelle]

Ready for Review, I added that change =0