search

codalab / codabench

Codabench is a flexible, easy-to-use and reproducible benchmarking platform. Check our paper at Patterns Cell Press https://hubs.li/Q01fwRWB0
Apache License 2.0
60 stars 26 forks source link

Feature request - "Cancel Submissions" #1487

Open johanneskruse opened 2 weeks ago

johanneskruse commented 2 weeks ago

Hi,

Thank you for the awesome platform.

I'm running a competition, and due to some technical issues, I've had to cancel and resubmit a lot of submissions.

Hence, a feature suggestion would be to add a "Cancel Submissions" option similar to "Rerun/Delete Submissions".

image

Best, Johannes

Didayolo commented 2 weeks ago

Interesting suggestion. However, as long as the cancellation does not really prevents computation (#872), this is not that relevant.

johanneskruse commented 2 weeks ago

@Didayolo, so the cancellation doesn't really do anything?

Didayolo commented 2 weeks ago

@johanneskruse Actually it depends. If the submission is still in "Submitted" status, still not sent to a worker, then it is cancelled for real (from what I know). However, if it is already running, cancelling it won't stop the on going computation from the compute worker.

johanneskruse commented 1 week ago

Makes sense, thanks for the clarification. During the issues encountered with #1473 I usually cancel submissions when they get congested (and backlog becomes too large, and people started submitted the same submissions over and over again), also to make it visible for participants that it is no longer submitting.