Closed guarin closed 1 month ago
All modified and coverable lines are covered by tests :white_check_mark:
Project coverage is 81.77%. Comparing base (
e0799ae
) to head (843c250
).
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
Changes
Closes #1583
Fixes an issue where too many tokens were masked because the class token was included in the calculation for the number of masked tokens.
The new behavior is:
mask_class_token=True
: Calculate number of masked tokens based on full sequence length and potentially mask class tokenmask_class_token=False
: Calculate number of masked tokens based on sequence length without the class token and never mask the class tokenThe original MAE code also calculates the masking ratio without the class token, see: https://github.com/facebookresearch/mae/blob/efb2a8062c206524e35e47d04501ed4f544c0ae8/models_mae.py#L130
How was it tested?