gaopengcuhk / Pretrained-Pix2Seq

Replication of Pix2Seq with Pretrained Model
60 stars 5 forks source link

About your setting 'mask' equals to 'False' #4

Closed Williamongh closed 3 years ago

Williamongh commented 3 years ago

Hi, thanks for sharing! I am wondering that why you set all elements of mask equal to False in pix2seq.py?

        src, mask = features[-1].decompose() 
        assert mask is not None
        mask = torch.zeros_like(mask).bool()   
hanqiu-hq commented 3 years ago

In our previous experiments (with weaker augmentation and shorter training epochs), we found that removing the mask in the transformer leads to slight improvement when changing from relative coordinates to absolute coordinates.