search

THUNLP-MT / ModelCompose

Official code for our paper "Model Composition for Multimodal Large Language Models"
Apache License 2.0
15 stars 1 forks source link

Audio projection #1

Open manushree635 opened 4 months ago

manushree635 commented 4 months ago

Is there any reason for using a q former for audio projection layer instead of linear layer like for image

carboncoo commented 3 months ago

Thank you for your interest in our work. Regarding the choice of projector type for audio, as it's not the main focus of our work, we directly followed some previous works and chose to use QFormer. If you're interested, we did try training with a linear projector in early versions, and there was no significant difference observed between the two (although we did not conduct a comprehensive evaluation).