search

aiondemand / AIOD-rest-api

Services for the core of AIoD: Authentication and the metadata catalogue with REST API.
https://api.aiod.eu
MIT License
10 stars 7 forks source link

Docker compose permission error #224

Open Taniya-Das opened 9 months ago

Taniya-Das commented 9 months ago

Observed permission error For ex. "PermissionError: [Errno 13] Permission denied: '/logstash/config/config/logstash.yml'", in es_logstash_setup container.

Reproduction

Not completely sure how to reproduce, this seems to depend on how docker has been installed. On Jos' computer, all is fine. On Taniya, it isn't, neither is it on Rinjani or Agung servers. To reproduce: clone the project into a new directory and run docker compose up -d.

Actual

Permission denied errors on some services that need to write information to the data folder, or to the logstash folder. Some services, such as mysql, are fine.

Expected

No permission denied exceptions.

Manually circumventing the problem

You can circumvent the problem by manually changing the owners of the directories (this is done on rinjani and agung) or by manually changing the permissions on the directories. This is no real solution of course - it should just work out of the box!

Additional info

We tried to fix this by making sure the correct directories exist with the correct user (that's why we committed data/connectors/.gitkeep for instance). This makes sure that the directories are created by the same user that also cloned the repo - which doesn't have to be the user that docker expects...

PGijsbers commented 8 months ago

When running the elasticsearch container, I also had the same problem, which was fixed by making the data/elasticsearch folder owned by root:root (sudo chown -R root:root data/elasticsearch).

Then I encountered the permission error above, where changing ownership did not work (it seems that es_logstash_setup uses a different user). I had to give write permission to others: find logstash/ -type f -exec chmod 666 {} +

edit: it looks like es_logstash_setup is running as apprunner user.

PGijsbers commented 8 months ago

As es_logstash_setup is a short-lived service, I recommend we just run the container as root by adding user: root to the docker-compose.yaml for that container. Unless anyone is aware of an easy way to set up permissions for a docker user that doesn't interfere too much with access on the host machine. Alternatively, we could look at which files exactly the container is trying to access and more narrowly assign permissions.