why rank 0 comsumes more gpu memory than other ranks within single machine

[huangjundashuaige]

Where does the extra memory consumption come from? Or I just simply use it wrong?

[github-actions[bot]]

Marking as stale. No activity in 60 days. Remove stale label or comment or this will be closed in 7 days.

[github-actions[bot]]

Marking as stale. No activity in 60 days.