The self.level_embed paramter is never used

[timmeinhardt]

Your MaskedTransformerDecoder never applies the level encoding during the forward pass. Was this intentionally?

[rmarcuzzi]

Hi! We don't use it, we just forgot to remove it from the code.

[timmeinhardt]

Did you ablate whether adding/removing the level encoding has an effect on the performance?

[rmarcuzzi]

In our early experiments, adding it didn't change the performance but we didn't check in the final model.