search

HKBUNLP / Mr.Harm-EMNLP2023

Code for our EMNLP 2023 paper - Beneath the Surface: Unveiling Harmful Memes with Multimodal Reasoning Distilled from Large Language Models
Apache License 2.0
10 stars 2 forks source link

FHM result #3

Closed walker107 closed 7 months ago

walker107 commented 7 months ago

Hi,author.I have attempted to reproduce the results on the FHM dataset following the dataset settings and hyperparameters specified in the paper. However, the reproduced results have not been satisfactory, with accuracy and F1 score both hovering around 62%. Could you please advise on any specific considerations to be taken when running experiments on the FHM dataset?

DanielLin97 commented 7 months ago

Note that the Data Preprocessing of FHM is a bit different from Harm-C/P. Please refer to https://github.com/HKBUNLP/Mr.Harm-EMNLP2023/tree/main/Data#data-preprocess.

walker107 commented 7 months ago

Thank you so much for your prompt response! I'm eager to understand if the data preprocessing in this paper aligns with MaskPrompt's approach of simply concatenating the original text data with the entity and race information. I tried this way, but unfortunately, I'm only achieving 70% accuracy.