search

CUMLSec / trex

MIT License
90 stars 13 forks source link

Is it a bug in fairseq\criterions\trex.py #13

Open iamawhalez opened 2 years ago

iamawhalez commented 2 years ago

https://github.com/CUMLSec/trex/blob/7b2cabaecdaeb043da48d85a9016fed391ea75a5/fairseq/criterions/trex.py#L84

i think the code here is to calculate the loss of static_field(code), so the ignore_index should be byte_fields. Am i right?

peikexin9 commented 2 years ago

Hi @iamawhalez, it is trying to ignore the loss on padding tokens in static code, so that's why it is ignoring padding. byte_fields won't participate in this loss computation. But your understanding is right, here is to calculate the loss for static field.