Open thevasudevgupta opened 3 years ago
There is small bug in following: https://github.com/google-research/bigbird/blob/103a3345f94bf6364749b51189ed93024ca5ef26/bigbird/pretrain/run_pretraining.py#L494
It should be self._trainable_weights = (self.extra_layer.trainable_weights +
self._trainable_weights = (self.extra_layer.trainable_weights +
There is small bug in following: https://github.com/google-research/bigbird/blob/103a3345f94bf6364749b51189ed93024ca5ef26/bigbird/pretrain/run_pretraining.py#L494
It should be
self._trainable_weights = (self.extra_layer.trainable_weights +