[ ] how effective are different compression? This will require passing some argument from train.py
[ ] given the same number of bytes communicated, which coding is most effective? This will require looking at gradient variance increase (already coded)
Only do this for one network, all algorithms. Do not show over iterations (no tuning step)
[x] See how rank changes with reshaping
Only do this for one network, all algorithms. Do not show over iterations (no tuning step).