The Conformer audio encoder:
The Conformer architecture is readily available via torchaudio, and only a few additional modules were coded.
Changes to the CoCa code which allow the Conformer encoder and the audio modality to be used with the CoCa architecture:
These changes include renaming and introducing a few variables and defining usage for them, as well as, slightly modifying the forward pass logic.
These commits essentially bring in two things:
The Conformer audio encoder: The Conformer architecture is readily available via
torchaudio, and only a few additional modules were coded.Changes to the CoCa code which allow the Conformer encoder and the audio modality to be used with the CoCa architecture: These changes include renaming and introducing a few variables and defining usage for them, as well as, slightly modifying the forward pass logic.