search

beeldengeluid / dane-example-worker

MIT License
0 stars 0 forks source link

Potential fixes to run worker on test input #20

Closed greenw0lf closed 3 months ago

greenw0lf commented 3 months ago

Linked to #19

I managed to make the worker run on test input. The error I am getting now (which I think it's fine for test input) is that the output cannot be uploaded to S3.

I removed /data and /config ignores in .dockerignore, as well as explicitly copied the config.yml file to /root/.DANE inside the Dockerfile.

I know data and config.yml should normally NOT be copied if the Docker image is published, but since this is an example worker meant as a starting template and for development/testing purposes only, this is acceptable.

I suggest making some sort of configuration for Docker such that you can have data and config.yml when testing, but in prod ignore them.

greenw0lf commented 3 months ago

Maybe we can have different .dockerignore files per branch and have prod and dev branches or something similar?

Veldhoen commented 3 months ago

We don't usually work with separate dev and prod branches, so although it may be a solution, it doesn't suit our way of working I'm afraid. Would it be a solution to bind mount ./config and ./data as volumes in the container? That allows you to simply add them at run time, without cluttering the image. You could then specify the instructions for doing so in the README a/o in the Wiki. See https://docs.docker.com/storage/bind-mounts/

greenw0lf commented 3 months ago

Ok, I did as you suggested and it indeed worked. I will revert the Docker changes and add instructions about how to test in both the README and the wiki

greenw0lf commented 3 months ago

@Veldhoen could you review the README explanation and let me know what you think?

Also, I think I will skip adding info to wiki as it will be duplicate.

Could you also check the comment I left on issue #3 ?