georgymh / decentralized-ml

Interoperable and decentralized machine learning.
Apache License 2.0
9 stars 5 forks source link

Optimizer transform split #34

Closed kiddyboots216 closed 5 years ago

kiddyboots216 commented 5 years ago

This PR integrates TRANSFORM_SPLIT into the Fed. Avg. Optimizer by updating the Optimizer to return an array of Jobs, updating the Communication Manager to run that array of Jobs, and updating all the tests so that they are ok with this new behavior. It's worth noting that the transform_function absolutely should not be on master. But that should be addressed in a separate PR if and when we make the decision to remove it; for now, I've added a few lines in _transform_and_split_job in the Runner to handle that.

georgymh commented 5 years ago

Minor things:

I didn't thoroughly check the test files changed by this PR but besides what I mentioned above, I think the other files look good. Neel, if you're able to, please review the tests for Panda.

kiddyboots216 commented 5 years ago
kiddyboots216 commented 5 years ago

@georgymh if you can approve the patch or give all necessary by tmrw 4:30 PM that would be ideal, as I'll have another few hours to work tomorrow and if this is merged I can get communication-job ready for review.

georgymh commented 5 years ago

With the last point I meant automating the clean up within the tests.

I have some time after class. I will try.