search

fac2003 / perceiver-multi-modality-pytorch

Implementation of Perceiver, General Perception with Iterative Attention, in Pytorch
MIT License
37 stars 4 forks source link

Gelu and pooling #6

Closed fac2003 closed 3 years ago

fac2003 commented 3 years ago

Adding support for GELU activation (to better match Perceiver preprint). Add method to override pooling and MultiModalityPerceiverNoPooling implementation that returns hidden representation without pooling or classifier (useful for multitask applications).