search

thuiar / MMSA

MMSA is a unified framework for Multimodal Sentiment Analysis.
MIT License
645 stars 104 forks source link

Could you please share the hyperparameters/configure files that produce the results in "results/result-stat.md"? #6

Closed NicholasGote closed 3 years ago

NicholasGote commented 3 years ago

Dear Authors,

Could you please share the hyperparameters/configure files that produce the results in "results/result-stat.md"?

I tried the default configuration files in this repo and I also tried to search the optimal configure using the fine-tune function, but I cannot reproduce the good results you displayed in the "results/result-stat.md" (I only tried it on the MOSI dataset currently).

Thank you very much for your support in advance.

iyuge2 commented 3 years ago

Hi, @NicholasGote. Sorry, but if I remember correctly, the default hyperparameters in our repo are exactly the parameters which can reproduce the results in "results/result-stat.md". In the following days, I will check that again. After that, I will inform you quickly. Thanks for your attention.

NicholasGote commented 3 years ago

Thank you very much. Here is more information:

The data I used is downloaded from "/raw_datasets/processed_data/cmu-mosi/seq_length_50" at the CMU_SDK ftp. I changed the feature dims of MOSI from (768, 5, 20) into (300, 5, 20) to match the downloaded data, and then run TFN model with default hyperparameters and got Has0_acc_2 around 71 (repeat 5 times as your setting) in the "mosi-regression.csv" files.

iyuge2 commented 3 years ago

@NicholasGote Thanks for your waiting!

The text features we used are extracted from pre-trained berts, different from the CMU_SDK. We also checked our hyperparams, which are exactly the parameters which can reproduce the results in "results/result-stat.md".

Additionally, for the convenience in reproducing our results, we have updated download links for all datasets features and pre-trained berts in README.md.

Best wishes!

NicholasGote commented 3 years ago

Thank you very much for the explanation, that solved my problem! The 'aligned_50.pkl' file in the MOSI folder seems to be truncated "_pickle.UnpicklingError: pickle data was truncated". Really appreciate if you could also check and update this one!

iyuge2 commented 3 years ago

Where did you download it from ？

NicholasGote commented 3 years ago

@iyuge2 From google driver. The file 'unaligned_50.pkl' works fine, while 'aligned_50.pkl' cannot be loaded with 'pickle.load'.

iyuge2 commented 3 years ago

@NicholasGote Sorry for our mistake. Now, we update the 'aligned_50.pkl' in google driver. Also, we add the sha-1 hash value for all features in our README.md. Thank you~

NicholasGote commented 3 years ago

It works. Thank you very much for your prompt support!