search

FedML-AI / FedNLP

FedNLP: An Industry and Research Integrated Platform for Federated Learning in Natural Language Processing, Backed by FedML, Inc. The Previous Research Version is Accepted to NAACL 2022
223 stars 45 forks source link

[SQuAD] crashed when doing official evaluation #11

Closed chaoyanghe closed 3 years ago

chaoyanghe commented 3 years ago

in eval_model_by_offical_script if n not in custom_parameter_names and not any(nd in n for nd in no_decay) IndexError: list index out of range

chaoyanghe commented 3 years ago

@yuchenlin @RaymondTseng Hi Yuchen & Zihang, do you know what causes this issue?

chaoyanghe commented 3 years ago

File "/home/chaoyanghe/sourcecode/PipeTransformer/examples/question_answering/question_answering_model.py", line 617, in eval_model_by_offical_script if n not in custom_parameter_names and not any(nd in n for nd in no_decay) IndexError: list index out of range

yuchenlin commented 3 years ago

can you make this bug more clearer? i thought you were talking about our FedNLP code....

chaoyanghe commented 3 years ago

we already discussed this. FedNLP also has this issue, we fixed this by using a mapping dictionary.

chaoyanghe commented 3 years ago

Since only SQuAD has the original unique ID, it is ok to me to make a special code patch. Thus we don't need further modification in a systematical way, and this issue can be closed.