Make DPMultiheadAttention drop-in compatible with nn.MultiheadAttention - Githubissues

pytorch / opacus

Training PyTorch models with differential privacy

https://opacus.ai

Apache License 2.0

1.67k stars 332 forks source link

Make DPMultiheadAttention drop-in compatible with nn.MultiheadAttention #528

Closed Wei-1 closed 1 year ago

Wei-1 commented 1 year ago

Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Docs change / refactoring / dependency upgrade

Motivation and Context / Related issue

This PR is target to resolve #123 by having an additional re-naming mechanism to match the state_dict structure of nn.MultiheadAttention.

How Has This Been Tested (if it applies)

Run pytest within the opacus folder:

============= 162 passed, 41 skipped, 4411 warnings in 243.18s (0:04:03) =============

Checklist

[x] The documentation is up-to-date with the changes I made.
[x] I have read the CONTRIBUTING document and completed the CLA (see CONTRIBUTING).
[x] All tests passed, and additional code has been covered with new tests.

Wei-1 commented 1 year ago

Remove for duplicated PR