Open kdmalc opened 1 year ago
While the decoders are different (different norms, have varying distances from each other, etc... we still don't have a good metric to meaningfully cluster the decoders yet... most distance metrics appear arbitrary wrt performance), I have shown that you can actually just swap the decoders from different clients and get the same performance, more or less (eg it's averaged out). This is still a preliminary result, may be inconsistent with CPHS
For both FedAVG and APFL. If local decs are very different then we have a case for personalization. If they are all more or less the same then maybe there is no personalization and there will only be a single optimal for all users.