Closed Didayolo closed 2 years ago
Also, what is this for?
In .env:
# Location to store submissions/cache -- absolute path!
HOST_DIRECTORY=/your/path/to/codabench/storage
In the setup command:
-v /your/path/to/codabench/storage:/codabench \
EDIT: simply create a folder and reference it as HOST_DIRECTORY
. This folder will be shared between the container and the host (compute worker).
UPDATE: for a GPU worker, I tried to manually change rabbit to www.codabench.org
Still not working:
[2022-08-31 13:08:45,434: ERROR/MainProcess] consumer: Cannot connect to amqp://2e6b227b-c960-44c4-8104-505e9f45077f:**@www.codabench.org:5672/11401520-1b5e-4290-bc2c-6f17526c0343: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1108).
Trying again in 2.00 seconds...
EDIT: Finally, by removing BROKER_USE_SSL=True
from the .env
file, the worker is connected to the queue:
[2022-08-31 14:15:08,167: INFO/MainProcess] Connected to amqp://2e6b2[...]-c960-44c[...]:**@www.codabench.org:5672/1140152[...]
[2022-08-31 14:15:08,180: INFO/MainProcess] mingle: searching for neighbors
[2022-08-31 14:15:09,220: INFO/MainProcess] mingle: all alone
[2022-08-31 14:15:09,258: INFO/MainProcess] compute-worker@37ca1eecf812 ready.
@dtuantran @bbearce
What about this? It is working now, right?
[image: image.png]
I see on FF that codabench.org is working. Is that what you meant? I also see master is passing!!! :)
On Wed, Oct 5, 2022 at 4:38 AM Adrien Pavão @.***> wrote:
@dtuantran https://github.com/dtuantran @bbearce https://github.com/bbearce
What about this? It is working now, right?
— Reply to this email directly, view it on GitHub https://github.com/codalab/codabench/issues/690#issuecomment-1268125924, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB2LN36N3EZFXP4M2VNB43LWBU47VANCNFSM5R3MNFXA . You are receiving this because you were mentioned.Message ID: @.***>
@bbearce
I was thinking of the setup of compute workers. Is it working fine now? For CPU and GPU? And by just following the wiki instructions?
Thanks
I had no trouble with the wiki for deploying these. Seems Tuan and Anne-Catherine had issues behind the firewall which needed special care.
On Thu, Oct 6, 2022 at 8:34 AM Adrien Pavão @.***> wrote:
@bbearce https://github.com/bbearce
I was thinking of the setup of compute workers. Is it working fine now? For CPU and GPU? And by just following the wiki instructions?
Thanks
— Reply to this email directly, view it on GitHub https://github.com/codalab/codabench/issues/690#issuecomment-1269960239, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB2LN35F7CAIZFBYC6BEOJ3WB3BNZANCNFSM5R3MNFXA . You are receiving this because you were mentioned.Message ID: @.***>
I've improved the documentation.
Maybe the remaining problem is the BROKER URL generated by CodaBench:
Shouldn't the domain appears somewhere here? Like
codabench.org
instead ofrabbit
?
So I agree. In docker-compose.yml under compute_worker service we see:
BROKER_URL=pyamqp://${RABBITMQ_DEFAULT_USER}:${RABBITMQ_DEFAULT_PASS}@${RABBITMQ_HOST}:${RABBITMQ_PORT}//
I think it would be better if we used:
BROKER_URL=pyamqp://${RABBITMQ_DEFAULT_USER}:${RABBITMQ_DEFAULT_PASS}@${DOMAIN_NAME}:${RABBITMQ_PORT}//
Also we should look into the code that generates BROKER_URLs when making queues manually and have it use the DOMAIN_NAME as well. I'm curious if anyone thinks this is a bad idea? One potential issue is if rabbit doesn't publish it's ports it may be unreachable but I don't think that is the case as I see in the "rabbit" service ports are being published. This makes sense as rabbit is reachable from other VMs.
If you have a DOMAIN_NAME available, it should be a good idea to generate the BROKER_URL from it in the queue management page. For now, I have edited the Queue Management wiki page to indicate how the copy-pasted BROKER_URL should be modified ( codabench.org
instead of rabbit
).
I have trouble creating compute workers.
Shouldn't the domain appears somewhere here? Like
codabench.org
instead ofrabbit
?I get the following error:
Or, when I edit the broker URL by hand: