search

google-research / bigbird

Transformers for Longer Sequences
https://arxiv.org/abs/2007.14062
Apache License 2.0
563 stars 101 forks source link

bug in line-494 of script- run_pretraining.py #12

Open thevasudevgupta opened 3 years ago

thevasudevgupta commented 3 years ago

There is small bug in following: https://github.com/google-research/bigbird/blob/103a3345f94bf6364749b51189ed93024ca5ef26/bigbird/pretrain/run_pretraining.py#L494

It should be self._trainable_weights = (self.extra_layer.trainable_weights +