Closed stas00 closed 3 years ago
@stas00
Can you please write a test code for finetune.sh.
As you can see I'm waiting for this ticket to be addressed before I'm able to write the tests.
Perhaps you can address that, and then I will have all the info needed to write the tests.
Until then please file a normal issue about it. I haven't done any rag work yet, so that's why I'm asking for support.
@lhoestq is working on this at the moment :-)
Actually I'm working on the finetuning script example, not eval ;) But maybe this can help with adding a test for the eval script example.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
stale
I'm going to try to write tests for
examples/rag
(https://github.com/huggingface/transformers/issues/7715), but first I'm trying to figure out how it works.Would it be possible to add a full
End-to-end evaluation
invocation example in https://github.com/huggingface/transformers/blob/master/examples/rag/README.md#end-to-end-evaluation? i.e. with the correct data.I tested https://github.com/huggingface/transformers/blob/master/examples/rag/README.md#retrieval-evaluation and it worked, but if I try to adapt the same params for e2e it crashes with:
I think it needs a different input data.
And we need 2 functional examples: for
qa
andans
each.I can handle adding this to the doc if you tell me what to add.
Thanks.
@patrickvonplaten, @lhoestq