envicutor / drafts

Issues and drafts for Envicutor
1 stars 0 forks source link

Contest simulation system #7

Closed Brikaa closed 3 months ago

Brikaa commented 4 months ago

Create a system that can use data found in #5 to simulate a contests in about two minutes. The system will have the test cases of the problems of each contest and sample submissions with their timestamp. It should simulate submitting the submissions in their original order (though with closer timestamps since we want the entire thing to run in about two minutes). Note that the execution and response order is not guaranteed in Envicutor, nor is the response order guaranteed in the network, so this has to be handled. That is, a user who submitted a valid submission before another user has to be ranked before them.

While the contest is running, the system should also simulate lots of malicious submissions (infinite loops, system integrity attacks, excessive memory allocation, etc.)

If no data was found out of #5, or if the following way is simpler, we can use our vjudge submissions from the problem solving course. We can pick ~12 problems, pick samples of our submissions that passed, samples that failed and samples that exceeded limits. We can use these submissions as a base for generating lots of other submissions (about 200 - 500). Other submissions can be generated via injecting slow code to make sure time-limit is exceeded, adding syntax errors, adding random comments to give a variety of submissions, etc.

The same malicious submissions idea also applies for this way.

The results of the contest (ranking, how many accepted submissions, how many unaccepted, etc.) shall be known before the simulation. After the simulation, the system shall compare the known results with the actual results to see if Envicutor was able to execute the submissions successfully, to time out long-running submissions, etc.

This system shall probably be developed in its own repo.

Preferably in something lightweight like NodeJS since we want to run this along with the code execution system on the same machine during the presentation. The system shall handle failures that could happen in Envicutor, and do its best to reliably carry out the simulation from start to finish in spite of errors.

Brikaa commented 3 months ago

Preferably in something lightweight like NodeJS since we want to run this along with the code execution system on the same machine during the presentation.

AbsoluteZero000 commented 3 months ago

k

Brikaa commented 3 months ago

Would be good if the status (ranking, no of accepted submissions, etc.) of the contest updated in real-time.

AbsoluteZero000 commented 3 months ago

u mean in the frontend?

Brikaa commented 3 months ago

Ye