search

ddlBoJack / emotion2vec

[ACL 2024] Official PyTorch code for extracting features and training downstream models with emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representation
576 stars 42 forks source link

Request for demo on using emotion2vec with Speech + Text modality #42

Open misaka14514 opened 1 month ago

misaka14514 commented 1 month ago

Hello there! I'm currently trying to use the emotion2vec for sentiment analysis tasks and appreciate your work. After reading related papers and documentation, I noticed that you have provided instructions on how to predict using speech or text modal data separately.  However, I am also interested in understanding how to combine both speech and text data (i.e., Speech + Text) for multimodal emotion prediction. According to my findings from literature, this seems like an important application scenario.  Therefore, could you please provide a simple example demonstrating how to integrate these two modalities of data and run the model? I believe this would be highly beneficial for other users as well.  Thank you! 97b35cdff111d3584459175d8e3b9b09

June1124 commented 1 month ago

I was wondering the same thing. Any results yet, please?

ddlBoJack commented 3 weeks ago

You can refer to Shi et al.'s(2020) and (2023) papers. We reproduced their methods to align with their numbers.