search

cpinitiative / online-judge

Serverless Online Judge
Mozilla Public License 2.0
17 stars 4 forks source link

Support for dynamic test case outputs #12

Open Yash-Singh1 opened 1 year ago

Yash-Singh1 commented 1 year ago

Some test cases can have multiple outputs such as https://cses.fi/problemset/task/1640. Currently, I believe that the judge doesn't validate dynamic outputs looking at the source code:

https://github.com/cpinitiative/online-judge/blob/74f57da422ea32899b1f1c1fd9871e4b92d119b6/submit/src/problemSubmission/createSubmission.ts#L307

Also, submitting to it on one of my USACO Guide Groups gives me partial WA while CSES gives me all AC.

megargayu commented 1 year ago

An idea to fix this could be replacing .out files for all problems with multiple correct answers and instead replacing it with an executable or other script file which would validate an output, and then automatically detecting a lack of a .out file in the actual grader code. Probably would have to start changing the code below: https://github.com/cpinitiative/online-judge/blob/74f57da422ea32899b1f1c1fd9871e4b92d119b6/submit/src/problemSubmission/fetchProblemTestCases.ts#L50-L61 Changing the Error thrown to instead check for an executable/script file and then if found set the necessary parameters so it can be used during grading.