search

alibaba / FederatedScope

An easy-to-use federated learning platform
https://www.federatedscope.io
Apache License 2.0
1.27k stars 207 forks source link

The calculation of the message bytes may not be accurate. #677

Closed zza234s closed 7 months ago

zza234s commented 1 year ago

Describe the bug The "count_bytes" function could not return the expected result when a message's content includes a tensor object. https://github.com/alibaba/FederatedScope/blob/6dfe8d4474451c07cd7d69690a2df33a1f455f19/federatedscope/core/message.py#L259-L269

To Reproduce image

I think the problem may be in the "asizeof" function. https://github.com/alibaba/FederatedScope/blob/6dfe8d4474451c07cd7d69690a2df33a1f455f19/federatedscope/core/message.py#L265

This function appears to inaccurately calculate the memory usage for pytorch tensor objects. For instance, it may return the same byte count even for tensors with very different shapes. image

rayrayraykk commented 1 year ago

👍Thanks for the catching! We will fix it ASAP.

rayrayraykk commented 1 year ago

You can either use

sys.getsizeof(t.storage())

or

t.element_size() * t.nelement()

to a tensor to get memory usage.