Open yaoxiao1999 opened 1 month ago
Hi Yao, I'm wondering have you fine-tuned BERT or RoBERTa? In this project, the prompt-based AD detection paradigm only works when I used BERT or RoBERTa that were fine-tuned on ADReSS training data. The fine-tuning codes are included in this repo as well. If you are actually running it on the fine-tuned BERT / RoBERTa model, do you mind checking the gradient when you fine-tune them to see if the gradient actually propagates to the model?
Hi, thank you for replying. I made sure that do_training is True and the logs also suggested that fine-tuning happened.
Or do you mean that I should fine-tune BERT/RoBERTA before prompt-based fine-tuning? i.e., in the off_line_model_dir
I should provide the fine-tuned model? If that's true, could you direct me to where the fine-tuning code is please as I struggle to find it.
Thank you!
I double-checked my code, and realised that I removed some hard-coded local model path for the simplicity of the codebase when putting the code on GitHub ... so when after the fine-tuning, when you conduct inference using the scripts run_prompt_finetune_test.py
, you may want to change the following two lines in the script prompt_finetune.py
,
model_dict = {'bert-base-uncased': os.path.join(args.off_line_model_dir, 'bert-base-uncased'),
'roberta-base': os.path.join(args.off_line_model_dir, 'roberta-base'),}
into lines like this:
model_dict = {'bert-base-uncased': off_line_model_dir + 'bert-base-uncased',
'bert-tuned28': off_line_model_dir + 'bert_post_train/epoch28',
'bert-tuned29': off_line_model_dir + 'bert_post_train/epoch29',
'bert-tuned30': off_line_model_dir + 'bert_post_train/epoch30',}
Here I stored the fine-tuned models in the off_line_model_dir
folder's subdirectory bert_post_train
, so you want to specify your own fine-tuned model directory in model_dict
, and specify the --model_name
argument to be the corresponding key in the model_dict
. (For example if I want to use the 28th epoch fine-tuned model stored at bert_post_train/epoch28
, I need to set --model_name bert-tuned28
.
But except for the above, you probably don't need to fine-tune BERT / RoBERTa before the prompt-based fine-tuning. Pls see if this solve the problem, and sorry about the inconvenience.
Hello,
The outputs of both
run_prompt_finetune.py
andrun_prompt_finetune_test.py
showed that the models always predicted positive labels. I tried both BERT and RoBERTa as the PLM.There's this warning but I doubt it's the cause of the problem:
UndefinedMetricWarning: Precision is ill-defined and being set to 0.0 in labels with no predicted samples. Use zero_division parameter to control this behavior.
Here are some of the outputs:
roberta-base_tempmanual1_verbmanual_full_100/version_2/checkpoints/epoch7/test_class_report.csv
roberta-base_tempmanual1_verbmanual_full_100/version_2/checkpoints/epoch7/test_results.csv
bert-base-uncased_tempmanual1_verbmanual_full_100/version_1_val/checkpoints/epoch9/test_class_report_cv1_fold8.csv
bert-base-uncased_tempmanual1_verbmanual_full_100/version_1_val/checkpoints/epoch9/test_results_cv1_fold8.csv
Could you please help? Thank you!