impossible to use a local worker

[dagecc-challenge]

Due to the public queue congestion, we have  issues with some evaluations during the final phase of our competition (ending tonight!), and we tried to restart a local worker that had started successfully a couple of months ago. Unfortunately, when our local worker starts ingesting submissions, we get an error :

WS: b'exec /opt/conda/bin/python: operation not permitted\n'

We tried installing another worker from scratch and we got the exact same thing. Do you have any idea about what could go wrong here?

Thanks

[Didayolo]

Hi @dagecc-challenge,

We recently updated the compute worker docker image, maybe the problem comes from this. To setup, you used this command, right?

docker run \
    -v /codabench:/codabench \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -d \
    --env-file .env \
    --name compute_worker \
    --restart unless-stopped \
    --log-opt max-size=50m \
    --log-opt max-file=3 \
    codalab/competitions-v2-compute-worker:latest 

Please try using this one (the docker tag is different):

docker run \
    -v /codabench:/codabench \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -d \
    --env-file .env \
    --name compute_worker \
    --restart unless-stopped \
    --log-opt max-size=50m \
    --log-opt max-file=3 \
    codalab/competitions-v2-compute-worker:cpu1.0 

[dagecc-challenge]

Thank you for your answer.

Unfortunately, using the alternative Docker tag did not resolve the issue. However, I re-installed Docker and am now encountering a different error:

python: can't open file '/app/program/ingestion.py': [Errno 2] No such file or directory.

Do you have any suggestions on how to address this new error?

[Didayolo]

And when you run the same submissions on the default queue, it is working fine?

[dagecc-challenge]

Yes, the problem is that the submission reachs the execution time limit (1200s). That's why we need to use our queue.

[dagecc-challenge]

The problem has been solved. There was an error in our configuration file. Thank you very much.