search

NZOI / nztrain

The NZOI online judge and training site written in Ruby
18 stars 13 forks source link

Add support for custom evaluators using compiled langauges #260

Closed Holmes98 closed 5 months ago

Holmes98 commented 9 months ago

This adds an optional language field to the Evaluator class. All existing/new evaluators will default to having no language, which maintains the existing functionality (script is executed as-is).

Note that C++ evaluators were already "supported" through cint.rb, but that is much slower as it requires the evaluator to be recompiled on every test case. That functionality can be considered deprecated and removed later, after fixing existing evaluators.

Also note that for evaluators with no language, Pygments will attempt to automatically detect the language based on the code (see here).

coveralls commented 9 months ago

Coverage Status

coverage: 37.481% (-0.01%) from 37.494% when pulling 8d84fcd1a7e25100eed3c07585798f83ed4a20a4 on compiled-evaluators into 4fc9595f09545893e9824ea07c78909a2a660bd3 on master.

BelgianSalamander commented 6 months ago

This is probably outside the scope of this pr but in future it may be good to not display the message This submission has not finished judging. Refresh this page in a minute or two to see the submission's score. when judging encounters an error (be it evaluator not compiling or some other error in the worker).

Holmes98 commented 6 months ago

Yeah, the submission page doesn't really handle judging errors. I've committed some improvements in 9e7aa3e.

Holmes98 commented 5 months ago

CI is currently failing; need to merge #266 first.

Holmes98 commented 5 months ago

Ugh, now coveralls is failing...

BelgianSalamander commented 5 months ago

Seems to be fixed now

Holmes98 commented 5 months ago

Deployed and converted all existing C++ evaluators that were previously using cint.rb to use this instead.