search

EBISPOT / ols4

Version 4 of the EMBL-EBI Ontology Lookup Service (OLS)
http://www.ebi.ac.uk/ols4/
Apache License 2.0
47 stars 22 forks source link

Docker indexing is not working #734

Closed rombaum closed 2 months ago

rombaum commented 2 months ago

Describe the bug The indexing process with a dataloader docker container fails since the user ols was added to the Docker file in the following commit https://github.com/EBISPOT/ols4/commit/9c623f2b562037b3e05adec9f45a3dd2e93db4fc.

To Reproduce Steps to reproduce the behavior:

  1. use the dev or stable branch
  2. export OLS4_CONFIG=./dataload/configs/efo.json or any other config file
  3. docker compose up --force-recreate --build --always-recreate-deps --attach-dependencies ols4-solr ols4-neo4j ols4-backend ols4-frontend
  4. See Operation not permitted error in the logs

Expected behavior Normally the ontology from the config file should be indexed into Neo4j.

Screenshots grafik log with the error message.

Upload minimal complete example If your ontology is not indexing correctly, please upload a minimal complete example ontology and related configuration that illustrates the problem.

Additional context I tried to index a local ontology via the docker compose command. I tried the indexing process afterwards with EFO which ends with the same issue. I checked out the following commit https://github.com/EBISPOT/ols4/commit/fafc7ed3fa111eacf4c39ff563bfa6d9daf53294 on my local machine and restarted the indexing which was working. The commit https://github.com/EBISPOT/ols4/commit/fafc7ed3fa111eacf4c39ff563bfa6d9daf53294 was done right before the commit https://github.com/EBISPOT/ols4/commit/9c623f2b562037b3e05adec9f45a3dd2e93db4fc which raises the issue. I copied the git history here to make it more clear:

grafik The indexing process in the commit on the bottom is working. The indexing process in the commit on the top is not working anymore.

I switched back than back to the current version of the branch dev and removed the changes of the commit https://github.com/EBISPOT/ols4/commit/9c623f2b562037b3e05adec9f45a3dd2e93db4fc in the Dockerfile of the dataloader. Afterwards I could successfully index EFO and also my local ontology from my machine.

I would guess that some sub directories of /opt/neo4j/ are generated after changing the owner to the user ols and that the owner of these directory is neo4j and the user ols doesn't have the rights by executing the scripts dataload.dockersh and create_neo4j_indexes.sh.

haideriqbal commented 2 months ago

@rombaum you would need to remove your old ols docker volumes and then rerun the whole process. This is happening because your old volumes were created with different user.

https://docs.docker.com/reference/cli/docker/volume/prune/

I'll add this is README troubleshooting section as well.

rombaum commented 2 months ago

I pruned the images and the container but not the volumes. I will have a look into this. Thanks!

rombaum commented 2 months ago

I still face the issue after I pruned the volumes via docker prune volume. I did the same by the way again for the container and the images.

haideriqbal commented 2 months ago

@rombaum did you check if the docker volumes were indeed removed when you pruned them? Asking this because with me it happens at times that volume were being used by some background process and were not deleted with normal prune command so have docker volume prune -af

let me know if that's the case otherwise, we can hop on a quick call to sort this out.

rombaum commented 2 months ago

That was the solution thanks.