search

lucidrains / performer-pytorch

An implementation of Performer, a linear attention-based transformer, in Pytorch
MIT License
1.07k stars 143 forks source link

Attention map #85

Closed merouone closed 2 years ago

merouone commented 2 years ago

Any idea about how to use BertViz or Tensor2Tensor visualization tool with PerformerLM class ? Thanks

lucidrains commented 2 years ago

@merouone it isn't possible, since linear attention doesn't manifest an attention map / matrix

merouone commented 2 years ago

@lucidrains Thanks.