Fix distributed training artifact and metrics logging

[maciej-sypetkowski]

Changelogs

• Fix wandb support when using distributed training (torchrun)
• sync metrics across distributed training processes.

---

Checklist:

• [x] Was this PR discussed in an issue? It is recommended to first discuss a new feature into a GitHub issue before opening a PR.
• [x] Add tests to cover the fixed bug(s) or the new introduced feature(s) (if appropriate).
• [x] Update the API documentation is a new function is added, or an existing one is deleted.
• [x] Write concise and explanatory changelogs above.
• [x] If possible, assign one of the following labels to the PR: feature, fix or test (or ask a maintainer to do it for you).

---

discussion related to that PR

[maciej-sypetkowski]

not passing lint check is coming from main

[codecov[bot]]

Codecov Report

Merging #480 (d1d47ae) into main (00979e2) will increase coverage by 0.01%. The diff coverage is 63.63%.

@@            Coverage Diff             @@
##             main     #480      +/-   ##
==========================================
+ Coverage   71.67%   71.68%   +0.01%     
==========================================
  Files          92       92              
  Lines        8454     8443      -11     
==========================================
- Hits         6059     6052       -7     
+ Misses       2395     2391       -4     

Flag	Coverage Δ
unittests	71.68% <63.63%> (+0.01%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
ipu	49.14% <ø> (ø)