wfondrie / depthcharge

A deep learning toolkit for mass spectrometry
https://wfondrie.github.io/depthcharge/
Apache License 2.0
59 stars 18 forks source link

change tgt_mask to boolean #38

Closed justin-a-sanders closed 1 year ago

justin-a-sanders commented 1 year ago

Current depthcharge peptide decoder converts tgt_mask from bool to float, allowing tokens to attend to the future during training. From the pytorch Transformer docs:

If a BoolTensor is provided, positions with True are not allowed to attend while False values will be unchanged. If a FloatTensor is provided, it will be added to the attention weight.

Leaving the mask as type Boolean resolves the issue.

wfondrie commented 1 year ago

Thanks @justin-a-sanders! Have you verified that this works on a GPU? I can't test that as part of our unit tests, but I want verify that it the mask is created on the correct device.

codecov[bot] commented 1 year ago

Codecov Report

Merging #38 (b987f92) into main (9bee6ca) will not change coverage. The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main      #38   +/-   ##
=======================================
  Coverage   91.48%   91.48%           
=======================================
  Files          19       19           
  Lines         963      963           
=======================================
  Hits          881      881           
  Misses         82       82           
Files Changed Coverage Δ
depthcharge/transformers/peptides.py 94.54% <100.00%> (ø)

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

justin-a-sanders commented 1 year ago

Thanks @justin-a-sanders! Have you verified that this works on a GPU? I can't test that as part of our unit tests, but I want verify that it the mask is created on the correct device.

Good call, needed to move it to the correct device.