Closed ngoel17 closed 2 years ago
Hi @ngoel17, thank you for your interest! We currently do not have plans to extend our implementation to BigBird, however, I'd suggest using the method from our second paper which does not require LRP (only gradient propagation).
Best, Hila.
Most of the NLP tasks use BigBird due to long text support. Can you please extend the class to support BigBird, by adding rules for torch.nn.modules.sparse.Embedding etc?