search

efficientqa / efficientqa.github.io

MIT License
3 stars 1 forks source link

Some sort of test run feedback would be helpful #5

Open mksenzov opened 4 years ago

mksenzov commented 4 years ago

An image that seem to run fine locally fails the test run upon submission. All the feedback we get is an error status in the table column on the dashboard. Is there a way to get some output that could help understand the issue?

mksenzov commented 4 years ago

As I go through it, it would be good to add --network="none" to first docker run in Testing submissions on https://efficientqa.github.io/submit.html

6, #7

but then in my case it was clearly not the only issue. An email with run log would go a long way here.

tomkwiat commented 4 years ago

Thanks for the feedback Mikhail.

I'm afraid it is difficult for us to automatically expose significant logs output while also maintaining a blind test set, so you will need to email nqopen@google.com and we will get back to you within a day.

I have looked up your logs and I suspect that there may be a bug stemming from our assumption that the Docker WORKDIR is set to /. Could you please retry your local test by calling ./submission.sh instead of /submission.sh.

Thanks also for the suggested addition to our Testing submissions section, I will add that now.

cheers, Tom