search

google-research / albert

ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Apache License 2.0
3.23k stars 570 forks source link

Description problem of attention_mask. #208

Open liushaoweihua opened 4 years ago

liushaoweihua commented 4 years ago

https://github.com/google-research/albert/blob/a41cf11700c1ed2b7beab0a2649817fa52c8d6e1/modeling.py#L838-L860

There may be a problem with the shape description of attention_mask in L857~L860, which should be [batch_size, from_seq_length].