Closed HuangZhe885 closed 1 year ago
Sorry for the late response. Can you paste your code of calculating volume here?
Sorry for the late response. Can you paste your code of calculating volume here? Thank you for your reply, I wrote this using the formula in the where2comm paper, but I'm not sure if it's correct.
non_zero_elements = communication_masks.nonzero().size(0)
communication_volume = math.log2(non_zero_elements * C * 32 / 8)
print("communication_volume",communication_volume)
Changing compression rate should work as it will reduce the C. Please check if you are giving the right C (i.e., the feature dimension after compression ) to your formula.
I am unable to reduce the communication volume even when adjusting the compression rate in where2comm. Could you help me?
Thank you for your excellent work. I use the formula log2 ( |M(k)| × D × 32/8) to calculate communication Volume, the calculation result is about 23~25. What should I do if I wish to manually lower this value? minimize the compression ratio or the feature size? I tried changing the compression ratio, but the Volume didn't go down.