gislipals commented 4 months ago

I get an error on the database build. The error is:

genai-stack-main-database-1 | Folder /data is not accessible for user: 7474 or group 7474. This is commonly a file permissions issue on the mounted folder.

I've tried running Docker and VSCode in Administrator mode, but that's not fixing the issue. Any ideas?

PS C:\Users\PC\Downloads\cim_genai\genai-stack-main> docker compose up --build time="2024-02-09T10:52:40-07:00" level=warning msg="The \"PWD\" variable is not set. Defaulting to a blank string." time="2024-02-09T10:52:40-07:00" level=warning msg="The \"PWD\" variable is not set. Defaulting to a blank string." time="2024-02-09T10:52:40-07:00" level=warning msg="The \"LANGCHAIN_PROJECT\" variable is not set. Defaulting to a blank string." time="2024-02-09T10:52:40-07:00" level=warning msg="The \"LANGCHAIN_API_KEY\" variable is not set. Defaulting to a blank string." time="2024-02-09T10:52:40-07:00" level=warning msg="The \"AWS_ACCESS_KEY_ID\" variable is not set. Defaulting to a blank string." time="2024-02-09T10:52:40-07:00" level=warning msg="The \"AWS_SECRET_ACCESS_KEY\" variable is not set. Defaulting to a blank string." time="2024-02-09T10:52:40-07:00" level=warning msg="The \"AWS_DEFAULT_REGION\" variable is not set. Defaulting to a blank string." time="2024-02-09T10:52:40-07:00" level=warning msg="The \"LANGCHAIN_PROJECT\" variable is not set. Defaulting to a blank string." time="2024-02-09T10:52:40-07:00" level=warning msg="The \"LANGCHAIN_API_KEY\" variable is not set. Defaulting to a blank string." time="2024-02-09T10:52:40-07:00" level=warning msg="The \"AWS_ACCESS_KEY_ID\" variable is not set. Defaulting to a blank string." time="2024-02-09T10:52:40-07:00" level=warning msg="The \"AWS_SECRET_ACCESS_KEY\" variable is not set. Defaulting to a blank string." time="2024-02-09T10:52:40-07:00" level=warning msg="The \"AWS_DEFAULT_REGION\" variable is not set. Defaulting to a blank string." time="2024-02-09T10:52:40-07:00" level=warning msg="The \"PWD\" variable is not set. Defaulting to a blank string." time="2024-02-09T10:52:40-07:00" level=warning msg="The \"LANGCHAIN_PROJECT\" variable is not set. Defaulting to a blank string." time="2024-02-09T10:52:40-07:00" level=warning msg="The \"LANGCHAIN_API_KEY\" variable is not set. Defaulting to a blank string." time="2024-02-09T10:52:40-07:00" level=warning msg="The \"AWS_ACCESS_KEY_ID\" variable is not set. Defaulting to a blank string." time="2024-02-09T10:52:40-07:00" level=warning msg="The \"AWS_SECRET_ACCESS_KEY\" variable is not set. Defaulting to a blank string." time="2024-02-09T10:52:40-07:00" level=warning msg="The \"AWS_DEFAULT_REGION\" variable is not set. Defaulting to a blank string." time="2024-02-09T10:52:40-07:00" level=warning msg="The \"PWD\" variable is not set. Defaulting to a blank string." time="2024-02-09T10:52:40-07:00" level=warning msg="The \"LANGCHAIN_PROJECT\" variable is not set. Defaulting to a blank string." time="2024-02-09T10:52:40-07:00" level=warning msg="The \"LANGCHAIN_API_KEY\" variable is not set. Defaulting to a blank string." time="2024-02-09T10:52:40-07:00" level=warning msg="The \"AWS_ACCESS_KEY_ID\" variable is not set. Defaulting to a blank string." time="2024-02-09T10:52:40-07:00" level=warning msg="The \"AWS_SECRET_ACCESS_KEY\" variable is not set. Defaulting to a blank string." time="2024-02-09T10:52:40-07:00" level=warning msg="The \"AWS_DEFAULT_REGION\" variable is not set. Defaulting to a blank string." [+] Building 2.8s (64/64) FINISHED => [genai-stack-main-bot internal] load build definition from bot.Dockerfile 0.0s => => transferring dockerfile: 519B 0.0s => [genai-stack-main-front-end internal] load build definition from front-end.Dockerfile 0.0s => => transferring dockerfile: 163B 0.0s => [genai-stack/pull-model:latest internal] load build definition from pull_model.Dockerfile 0.0s => => transferring dockerfile: 1.61kB 0.0s => [genai-stack-main-pdf_bot internal] load build definition from pdf_bot.Dockerfile 0.0s => => transferring dockerfile: 40B 0.0s => [genai-stack-main-loader internal] load build definition from loader.Dockerfile 0.0s => => transferring dockerfile: 549B 0.0s => [genai-stack-main-api internal] load build definition from api.Dockerfile 0.0s => => transferring dockerfile: 474B 0.0s => [genai-stack-main-bot internal] load .dockerignore 0.0s => => transferring context: 34B 0.0s => [genai-stack-main-front-end internal] load .dockerignore 0.0s => => transferring context: 34B 0.0s => [genai-stack/pull-model:latest internal] load .dockerignore 0.0s => => transferring context: 34B 0.0s => [genai-stack-main-pdf_bot internal] load .dockerignore 0.0s => => transferring context: 34B 0.0s => [genai-stack-main-loader internal] load .dockerignore 0.0s => => transferring context: 34B 0.0s => [genai-stack-main-api internal] load .dockerignore 0.0s => => transferring context: 34B 0.0s => [genai-stack-main-bot internal] load metadata for docker.io/langchain/langchain:latest 1.0s => [genai-stack-main-front-end internal] load metadata for docker.io/library/node:alpine 1.1s => [genai-stack/pull-model:latest] resolve image config for docker.io/docker/dockerfile:1.4 1.0s => [auth] langchain/langchain:pull token for registry-1.docker.io 0.0s => [auth] library/node:pull token for registry-1.docker.io 0.0s => [auth] docker/dockerfile:pull token for registry-1.docker.io 0.0s => [genai-stack-main-pdf_bot internal] load build context 0.0s => => transferring context: 14.03kB 0.0s => [genai-stack-main-api 1/8] FROM docker.io/langchain/langchain@sha256:72f8f54e130c711b17dd025d19c96204a2faf044d85caa0a31d7f1f5bff3b58d 0.0s => [genai-stack-main-loader internal] load build context 0.0s => => transferring context: 121.18kB 0.0s => [genai-stack-main-api internal] load build context 0.0s => => transferring context: 4.12kB 0.0s => [genai-stack-main-bot internal] load build context 0.0s => => transferring context: 5.29kB 0.0s => CACHED [genai-stack-main-api 2/8] WORKDIR /app 0.0s => CACHED [genai-stack-main-api 3/8] RUN apt-get update && apt-get install -y build-essential curl software-properties-common && rm -rf /var/lib/apt/lists/* 0.0s => CACHED [genai-stack-main-pdf_bot 4/8] COPY requirements.txt . 0.0s => CACHED [genai-stack-main-pdf_bot 5/8] RUN pip install --upgrade -r requirements.txt 0.0s => CACHED [genai-stack-main-pdf_bot 6/8] COPY pdf_bot.py . 0.0s => CACHED [genai-stack-main-pdf_bot 7/8] COPY utils.py . 0.0s => CACHED [genai-stack-main-pdf_bot 8/8] COPY chains.py . 0.0s => [genai-stack/pull-model:latest] exporting to image 0.1s => => exporting layers 0.0s => => writing image sha256:fec7189f9b97c5b67612708a3a8cddaecbeb1ce3487affe72a536da39ae7b27b 0.0s => => naming to docker.io/library/genai-stack-main-pdf_bot 0.0s => => writing image sha256:10dfc60a1969422a14678ad0ed2a471876ad3398e50875cb6eda27ae2788145a 0.0s => => naming to docker.io/library/genai-stack-main-bot 0.0s => => writing image sha256:f5443ba1df848c1a680ab0fd459934f18ab3fd2598e3dd648b6d534d619e39d8 0.0s => => naming to docker.io/library/genai-stack-main-loader 0.0s => => writing image sha256:170a2fca3c4af57b605e8c3653aec570f48b21357ef70dc8222d1dffe041d06e 0.0s => => naming to docker.io/library/genai-stack-main-api 0.0s => => writing image sha256:464569622e08e6a747b47a7beae8c9d31608e52988cbd771047722ca7edb3ff8 0.0s => => naming to docker.io/library/genai-stack-main-front-end 0.0s => => writing image sha256:8d582443f563545c3fddc5446ccde2b2b0e09e13927f1500e7fc1998b3122137 0.0s => => naming to docker.io/genai-stack/pull-model:latest 0.0s => CACHED [genai-stack-main-bot 4/8] COPY requirements.txt . 0.0s => CACHED [genai-stack-main-bot 5/8] RUN pip install --upgrade -r requirements.txt 0.0s => CACHED [genai-stack-main-bot 6/8] COPY bot.py . 0.0s => CACHED [genai-stack-main-bot 7/8] COPY utils.py . 0.0s => CACHED [genai-stack-main-bot 8/8] COPY chains.py . 0.0s => CACHED [genai-stack-main-loader 4/9] COPY requirements.txt . 0.0s => CACHED [genai-stack-main-loader 5/9] RUN pip install --upgrade -r requirements.txt 0.0s => CACHED [genai-stack-main-loader 6/9] COPY loader.py . 0.0s => CACHED [genai-stack-main-loader 7/9] COPY utils.py . 0.0s => CACHED [genai-stack-main-loader 8/9] COPY chains.py . 0.0s => CACHED [genai-stack-main-loader 9/9] COPY images ./images 0.0s => CACHED [genai-stack-main-api 4/8] COPY requirements.txt . 0.0s => CACHED [genai-stack-main-api 5/8] RUN pip install --upgrade -r requirements.txt 0.0s => CACHED [genai-stack-main-api 6/8] COPY api.py . 0.0s => CACHED [genai-stack-main-api 7/8] COPY utils.py . 0.0s => CACHED [genai-stack-main-api 8/8] COPY chains.py . 0.0s => CACHED [genai-stack/pull-model:latest] docker-image://docker.io/docker/dockerfile:1.4@sha256:9ba7531bd80fb0a858632727cf7a112fbfd19b17e94c4e84ced81e24ef1a0dbc 0.0s => [genai-stack-main-front-end internal] load build context 0.0s => => transferring context: 324.96kB 0.0s => [genai-stack-main-front-end 1/4] FROM docker.io/library/node:alpine@sha256:4cc2d9f365691fc6f8fe227321d32d9a2691216a71f51c21c7f02224515dea48 0.0s => CACHED [genai-stack-main-front-end 2/4] WORKDIR /app 0.0s => CACHED [genai-stack-main-front-end 3/4] COPY front-end/ . 0.0s => CACHED [genai-stack-main-front-end 4/4] RUN npm install 0.0s => [genai-stack/pull-model:latest internal] load .dockerignore 0.0s => [genai-stack/pull-model:latest internal] load build definition from pull_model.Dockerfile 0.0s => [genai-stack/pull-model:latest internal] load metadata for docker.io/babashka/babashka:latest 1.2s => [genai-stack/pull-model:latest internal] load metadata for docker.io/ollama/ollama:latest 1.2s => [auth] ollama/ollama:pull token for registry-1.docker.io 0.0s => [auth] babashka/babashka:pull token for registry-1.docker.io 0.0s => [genai-stack/pull-model:latest ollama 1/1] FROM docker.io/ollama/ollama:latest@sha256:3bc28f48a60ee34574dca0b0e310eff21e171b55d83fa06384bd83b97d9482b8 0.0s => [genai-stack/pull-model:latest stage-1 1/3] FROM docker.io/babashka/babashka:latest@sha256:a4900a2ff295705277732bdecd3f9dce249906858748cb31aad0b2917364128d 0.0s => [genai-stack/pull-model:latest internal] preparing inline document 0.0s => CACHED [genai-stack/pull-model:latest stage-1 2/3] COPY --from=ollama /bin/ollama ./bin/ollama 0.0s => CACHED [genai-stack/pull-model:latest stage-1 3/3] COPY <<EOF pull_model.clj 0.0s [+] Running 7/7

Container genai-stack-main-pull-model-1 Created 0.0s
Container genai-stack-main-database-1 Created 0.0s
Container genai-stack-main-bot-1 Recreated 0.1s
Container genai-stack-main-pdf_bot-1 Recreated 0.1s
Container genai-stack-main-api-1 Recreated 0.1s
Container genai-stack-main-loader-1 Recreated 0.1s
Container genai-stack-main-front-end-1 Recreated 0.1s Attaching to genai-stack-main-api-1, genai-stack-main-bot-1, genai-stack-main-database-1, genai-stack-main-front-end-1, genai-stack-main-loader-1, genai-stack-main-pdf_bot-1, genai-stack-main-pull-model-1 genai-stack-main-pull-model-1 | pulling ollama model gpt-4 using http://host.docker.internal:11434 genai-stack-main-pull-model-1 | OLLAMA model only pulled if both LLM and OLLAMA_BASE_URL are set and the LLM model is not gpt genai-stack-main-pull-model-1 exited with code 0 genai-stack-main-database-1 | genai-stack-main-database-1 | Folder /data is not accessible for user: 7474 or group 7474. This is commonly a file permissions issue on the mounted folder. genai-stack-main-database-1 | genai-stack-main-database-1 | Hints to solve the issue: genai-stack-main-database-1 | 1) Make sure the folder exists before mounting it. Docker will create the folder using root permissions before starting the Neo4j container. The root permissions disallow Neo4j from writing to the mounted folder. genai-stack-main-database-1 | 2) Pass the folder owner's user ID and group ID to docker run, so that docker runs as that user. genai-stack-main-database-1 | If the folder is owned by the current user, this can be done by adding this flag to your docker run command: genai-stack-main-database-1 | --user=$(id -u):$(id -g) genai-stack-main-database-1 | genai-stack-main-database-1 exited with code 1 dependency failed to start: container for service "database" exited (1)

hafiz031 commented 4 months ago

This can be fixed with sudo chmod 777 data. I don't know if this will have any side effects.

Yusful33 commented 4 months ago

Just an FYI but I had to restart Docker after running sudo chmod 777 data for it to take affect.

gislipals commented 4 months ago

This can be fixed with sudo chmod 777 data. I don't know if this will have any side effects.

Fixed in what way? That´s a Linux command and I´m on Windows.

hafiz031 commented 4 months ago

@gislipals Yes, in my case, it is fixed. And yeah, that's true @Yusful33.

internetscooter commented 4 months ago

I haven't found the solution but is it possible the "user" field for neo4j is being interpreted as the user for write permissions?

If I tried the following updates in docker-compose.yml, "user" was being flagged as being already defined. It seems suspect that 7474 is the user and that is defined a little after user on the database definition.

Permission denied when spinning up a neo4j container via colima

internetscooter commented 4 months ago

Further to this, if I comment out the docker-compose.yml, user part the database starts...

database: # user: neo4j:neo4j

database-1 | Warning: Folder mounted to "/data" is not writable from inside container. Changing folder owner to neo4j. database-1 | Installing Plugin 'apoc' from /var/lib/neo4j/labs/apoc-*-core.jar to /var/lib/neo4j/plugins/apoc.jar database-1 | Applying default values for plugin apoc to neo4j.conf database-1 | Changed password for user 'neo4j'. IMPORTANT: this change will only take effect if performed before the database is started for the first time. database-1 | 2024-02-25 21:47:49.226+0000 INFO Starting... database-1 | 2024-02-25 21:47:49.843+0000 INFO This instance is ServerId{f1053080} (f1053080-12ac-4837-a6af-3f43af9b7c40) database-1 | 2024-02-25 21:47:50.345+0000 INFO ======== Neo4j 5.11.0 ======== database-1 | 2024-02-25 21:47:53.501+0000 INFO Bolt enabled on 0.0.0.0:7687.

Setting...

database: user: neo4j

Seems to fix the problem (but I don't know what other problems it creates)...

database-1 | Installing Plugin 'apoc' from /var/lib/neo4j/labs/apoc-*-core.jar to /var/lib/neo4j/plugins/apoc.jar database-1 | Applying default values for plugin apoc to neo4j.conf database-1 | Changed password for user 'neo4j'. IMPORTANT: this change will only take effect if performed before the database is started for the first time. database-1 | 2024-02-25 21:55:09.778+0000 INFO Starting... database-1 | 2024-02-25 21:55:10.340+0000 INFO This instance is ServerId{f1053080} (f1053080-12ac-4837-a6af-3f43af9b7c40) database-1 | 2024-02-25 21:55:10.818+0000 INFO ======== Neo4j 5.11.0 ======== database-1 | 2024-02-25 21:55:13.480+0000 INFO Bolt enabled on 0.0.0.0:7687. database-1 | [main] INFO org.eclipse.jetty.server.Server - jetty-10.0.15; built: 2023-04-11T17:25:14.480Z; git: 68017dbd00236bb7e187330d7585a059610f661d; jvm 17.0.8.1+1 database-1 | [main] INFO org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.s.h.MovedContextHandler@29ff01df{/,null,AVAILABLE} database-1 | [main] INFO org.eclipse.jetty.server.session.DefaultSessionIdManager - Session workerName=node0 database-1 | [main] INFO org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.s.ServletContextHandler@5d88471a{/db,null,AVAILABLE} database-1 | [main] INFO org.eclipse.jetty.webapp.StandardDescriptorProcessor - NO JSP Support for /browser, did not find org.eclipse.jetty.jsp.JettyJspServlet database-1 | [main] INFO org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.w.WebAppContext@60d4269a{/browser,jar:file:/var/lib/neo4j/lib/neo4j-browser-5.11.0.jar!/browser,AVAILABLE} database-1 | [main] INFO org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.s.ServletContextHandler@5e206acb{/,null,AVAILABLE} database-1 | [main] INFO org.eclipse.jetty.server.AbstractConnector - Started http@24ab86eb{HTTP/1.1, (http/1.1)}{0.0.0.0:7474} database-1 | [main] INFO org.eclipse.jetty.server.Server - Started Server@30f1a181{STARTING}[10.0.15,sto=0] @4796ms database-1 | 2024-02-25 21:55:13.963+0000 INFO Remote interface available at http://localhost:7474/ database-1 | 2024-02-25 21:55:13.966+0000 INFO id: 1BCCDA4668AD2F121B7782D878B3F151FD938E8774C3F698DD9809D9C9429284 database-1 | 2024-02-25 21:55:13.967+0000 INFO name: system database-1 | 2024-02-25 21:55:13.967+0000 INFO creationDate: 2024-02-25T21:47:50.972Z database-1 | 2024-02-25 21:55:13.967+0000 INFO Started.

SAGE-Rebirth commented 3 months ago

I've also got the same issue.

Container genai-stack-pull-model-1 Exited 1.0s ✘ Container genai-stack-database-1 Error 0.0s

Container genai-stack-pdf_bot-1 Recreated 1.6s
Container genai-stack-loader-1 Recreated 1.6s
Container genai-stack-bot-1 Recreated 1.6s
Container genai-stack-api-1 Recreated 1.6s
Container genai-stack-front-end-1 Recreated 1.4s dependency failed to start: container genai-stack-database-1 exited (1)

internetscooter commented 3 months ago

@SAGE-Rebirth did you try changing user to neo4j as above? It worked for me for a while but starting a new instance seems to have stopped it. I am trying to fix myself.

SAGE-Rebirth commented 3 months ago

@internetscooter Yes I did then, I encountered some other issues and found that if you change ---> database: user: ${USER_ID}:${GROUP_ID}. Then it works. However the api-stack raises error.

internetscooter commented 3 months ago

@SAGE-Rebirth I got it to work by following the sequence.

First comment out

`database:

user: neo4j:neo4j`

Run docker compose build

Then remove the comment to:

database: user: neo4j

Run docker compose build again

This is obviously a dirty hack but does indicate what the issue is. Removing "user" stops it getting confused with an os user and allows the /data access to be done. Then adding it back allows the database to work.

The true fix will be in fixing up the config to remove the confusion and have it run smoothly (I don't know enough about docker though to spend the time to do this).

SAGE-Rebirth commented 3 months ago

@internetscooter Hey! thanks for the kind gesture but unfortunately it's not working. I'm figuring it out. If I get it working I'll definitely share my approach.

Apoc182 commented 3 months ago

@SAGE-Rebirth

I think what @internetscooter may have meant was docker compose up as opposed to docker compose build.

Running it once with it commented I assume (haven't taken the time to investigate exactly whats happening) allows the folder to be created and permissions to be set, then uncommenting it and running up again there is no error.

John-Holden commented 3 months ago

The above fixed it for me, commenting out and removing with docker compose up --build

noonething commented 2 months ago

My error was making a data folder in the host computers root folder, instead of in the GenAI home directory.

docker / genai-stack

Database build error during Docker Compose: Folder /data is not accessible. #123

user: neo4j:neo4j`