Optimize test_bounded_mean test execution

alejandrosame commented 3 years ago

Description

Added the possibility to write to file the serialized algorithm data. Currently, we have unit tests that take too much time to run, which hinders quick iteration (mentioned in https://github.com/OpenMined/PyDP/issues/265).

The current PR extends the Summary object to let us save and load protobuf data to a file in order to use it in fixtures. This has the benefit of paying for the creation of "big" setups only once, while the rest of the times the fixture will be loaded from a test data file.

With this approach, we can already see an important improvement for test_bounded_mean, where test execution time drops from ~124s to ~0.04s.

This is just a first step towards test optimization, future steps are:

Generalize fixture test to make it reusable. This is best done once another test similar to test_bounded_mean is introduced, to know how to properly refactor.
Optimize test_distributions. This test is the only one left that doesn't run in the order of centiseconds (currently ~35s). However, it seems to require a different serde strategy (most probable, with pickle).

This PR depends on #332 to be merged first.

Affected Dependencies

No new dependencies added.

How has this been tested?

Tests run locally

Checklist

[x] I have followed the Contribution Guidelines and Code of Conduct
[x] I have commented my code following the OpenMined Styleguide
[x] I have labeled this PR with the relevant Type labels
[x] My changes are covered by tests

alejandrosame commented 3 years ago

Now that #332 is merged, I can take out the WIP from this PR. If tests are successful, it will be ready for review @chinmayshah99 and @BenjaminDev.

chinmayshah99 commented 3 years ago

I think we need to update all our tests right?

alejandrosame commented 3 years ago

@chinmayshah99 I think they are ok, apart from the distributions test mentioned before (which anyway requires a different serde strategy). We can simply keep track if we add a new test with heavy setup like here. In that case, we can use the same fixture approach to keep test execution fast (and generalize code as we see it fits best).

OpenMined / PyDP