search

OpenMined / PySyft

Perform data science on data that remains in someone else's server
https://www.openmined.org/
Apache License 2.0
9.53k stars 1.99k forks source link

Benchmark FixedPrecision, LargePrecision and CRTPrecision #2734

Closed LaRiffle closed 4 years ago

LaRiffle commented 5 years ago

We want to test in real context and benchmark our 3 methods to convert float to fixed precision integers (which are used when encrypting tensors).

We'd like to do this with model evaluation. For example, let's do this on a simple model trained on MNIST (maybe just a 3-layer MLP).

So the job is to train this model, and then convert its parameters to fixed precision using the 3 strategy we currently support: FixedPrecision, LargePrecision and CRTPrecision (see the .fix_prec() api to learn how to do it)

Then, benchmark the runtime for evaluation for each of them.

This benchmark would be super valuable for the Crypto team and would help find hidden bugs!

The output of this project would be a jupyter notebook in 5 parts: 1) training the model in clear 2) -3-4) converting to the 3 fix precision strategies and monitor runtime of the evaluation + accuracy 5) Compare strategy pros and cons

refactormyself commented 5 years ago

Hello @LaRiffle, I will like to take this issue as my first issue.

LaRiffle commented 5 years ago

Sure go ahead!

NateSolon commented 4 years ago

Hey @refactormyself are you still working on this? I think this would be a great project for where I'm at right now - maybe we could both take a stab at it and compare?

NateSolon commented 4 years ago

@LaRiffle does the model need to be trained with federated learning or can I just train a model with vanilla PyTorch and then test the different precisions?

refactormyself commented 4 years ago

I am working on it.

On Sun, Dec 1, 2019 at 11:05 PM NateSolon notifications@github.com wrote:

Hey @refactormyself https://github.com/refactormyself are you still working on this? I think this would be a great project for where I'm at right now - maybe we could both take a stab at it and compare?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/OpenMined/PySyft/issues/2734?email_source=notifications&email_token=AEJIRHMOGSEQ7SYJC365YGDQWQYL5A5CNFSM4JNYAFC2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFRXEPA#issuecomment-560165436, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEJIRHLNLOQ5IK43DJWZPG3QWQYL5ANCNFSM4JNYAFCQ .