Closed Johnny785 closed 2 weeks ago
Hi @Johnny785, thanks for opening an issue!
So that we can best help you, could you:
transformers-cli env
in the terminal and copy-paste the outputAs this involves custom code and overriding standard methods, this is a question best placed in our forums. We try to reserve the github issues for feature requests and bug reports.
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.
Please note that issues that do not follow the contributing guidelines are likely to be ignored.
System Info
python version 3.8.5, transformers version 4.19.1
Who can help?
No response
Information
Tasks
examples
folder (such as GLUE/SQuAD, ...)Reproduction
Expected behavior
I customized my forward function to return an input field in the dictionary, which is of size 5BxC where B is the batch size. Other metrics such as loss and logits are BxC or B. However, in the compute_metrics function I customized, I received only BxC data entries from the EvalPrediction.predictions attribute for the input I returned in forward. Other fields that have the first dimension equal to B are passed in correctly. When I changed my input field shape to Bx5xC, it is also passed in correctly.
I read through the code that calls the compute_metrics function, however, I did not find a piece of code that trims the output returned from the model. After experimenting, I found that it's not taking mean, taking sum or taking max.
I would really appreciate an explanation as to why this happens, thank you for your assistance