search

declare-lab / hfusion

Multimodal sentiment analysis using hierarchical fusion with context modeling
MIT License
44 stars 22 forks source link

How unimodal.pickle was generated? #3

Open AlexTo opened 4 years ago

AlexTo commented 4 years ago

Hi May I know how unimodal.pickle was generated? Is there source code that produces this file? How do I interpret the dimension of "audio_train", "video_train" and "text_train" ? Thank you

SanaRahmani commented 4 years ago

Hi, you can do a trick, it seems like the first part of the model that generates unimodal pickle file is much like the same part in their previous paper "Context-Dependent Sentiment Analysis in User-Generated Videos" which is published here by the author: https://github.com/soujanyaporia/multimodal-sentiment-analysis so in the code above you preprocess data and then generate unimodal pickle file as the guide suggests you can use this file here in HFusion and by very few changes you can get it to run(dense layers are set to have 4 units while file generated throw my suggested way is expected to be 3) at the end this is not ideal and might even get you poor results but its a point you can start.... my results in this way where about 5 percent lower than accuracy reported in the paper for the same dataset. also why is accuracy this low even for their own pickle file provided, does anyone know what dataset this file is generated from?