Closed bracci-quest-it closed 1 year ago
Joffcom
commented
1 year ago Hey @bracci-quest-it,
Have you made any changes to the image? I assume there is a reason for using the custom image, I don't think you need the n8n start at the end either.
One thing worth checking is if your custom image runs without the database options set, The basic auth options can be removed as well as the default would be disabled.
bracci-quest-it
commented
1 year ago Have you made any changes to the image? yes I have, and pnpm dev runs fine without errors.
One thing worth checking is if your custom image runs without the database options set. The custom docker image doesn't run even without the database options set.
I have double checked and inside docker's inspect i can see all the envs.
Joffcom
commented
1 year ago Hey @bracci-quest-it,
It wasn't really checking the env options were set but more checking things that cause common issues. What are the changes you have made to the image? While pnpm dev might be all good it is worth checking that a change made isn't causing the issue.
Have you tried a custom image on a clean download of the code as well?
Joffcom
commented
1 year ago Hey @bracci-quest-it,
I have just given this a go on a clean pull of the n8n repo and for me this working as expected.
~/code/javascript/n8n-io-tmp/n8n | master .................. 7s | 01:53:16 pm
> docker build -t n8n-custom -f docker/images/n8n-custom/Dockerfile .
[+] Building 298.5s (27/27) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 1.80kB 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 191B 0.0s
=> [internal] load metadata for docker.io/n8nio/base:16 3.4s
=> [internal] load build context 1.3s
=> => transferring context: 24.27MB 1.3s
=> [builder 1/18] FROM docker.io/n8nio/base:16@sha256:5ed0cee768089444 14.1s
=> => resolve docker.io/n8nio/base:16@sha256:5ed0cee768089444ab0f77efb75 0.0s
=> => sha256:5ed0cee768089444ab0f77efb75323b5455185c5422 2.38kB / 2.38kB 0.0s
=> => sha256:4aaa86babb897d7e4681d70f01c80e0145dba095550 1.62kB / 1.62kB 0.0s
=> => sha256:6b13ed9707d08e1cd218fd5b59a0bd657c25034d044 7.29kB / 7.29kB 0.0s
=> => sha256:a9eaa45ef418e883481a13c7d84fa9904f2ec56789c 3.26MB / 3.26MB 0.8s
=> => sha256:f4b29973f215f1a2fa1660e079d82585c55c2c82 36.62MB / 36.62MB 11.7s
=> => sha256:dc47093e946419f96a64c8bda5bd725b3d28a3771ad 2.41MB / 2.41MB 4.3s
=> => extracting sha256:a9eaa45ef418e883481a13c7d84fa9904f2ec56789c52a87 0.3s
=> => sha256:9903ed942feca9a23884ff22ab7510ca7e3ed63e193e573 449B / 449B 1.2s
=> => sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d 32B / 32B 1.4s
=> => sha256:51f1f9c57035d2014c14000214725ec127d9e701ee947e5 229B / 229B 1.6s
=> => sha256:2c0b651d04a94571500f5d3133c8187408d2cea7 26.72MB / 26.72MB 12.2s
=> => extracting sha256:f4b29973f215f1a2fa1660e079d82585c55c2c829224bb35 1.0s
=> => extracting sha256:dc47093e946419f96a64c8bda5bd725b3d28a3771adea704 0.1s
=> => extracting sha256:9903ed942feca9a23884ff22ab7510ca7e3ed63e193e573e 0.0s
=> => extracting sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6 0.0s
=> => extracting sha256:51f1f9c57035d2014c14000214725ec127d9e701ee947e57 0.0s
=> => extracting sha256:2c0b651d04a94571500f5d3133c8187408d2cea7bb61c57b 0.8s
=> [builder 2/18] COPY turbo.json package.json .npmrc pnpm-lock.yaml pn 0.3s
=> [builder 3/18] COPY scripts ./scripts 0.0s
=> [builder 4/18] COPY packages ./packages 0.3s
=> [builder 5/18] COPY patches ./patches 0.0s
=> [builder 6/18] RUN apk add --update libc6-compat jq 1.2s
=> [builder 7/18] RUN corepack enable && corepack prepare --activate 2.6s
=> [builder 8/18] RUN chown -R node:node . 3.6s
=> [builder 9/18] RUN pnpm install --frozen-lockfile 43.4s
=> [builder 10/18] RUN pnpm build 170.2s
=> [builder 11/18] RUN rm -rf node_modules 5.8s
=> [builder 12/18] RUN jq 'del(.pnpm.patchedDependencies)' package.json 0.2s
=> [builder 13/18] RUN jq '{name: .name, version: .version}' packages/ed 0.2s
=> [builder 14/18] RUN jq '{name: .name, version: .version}' packages/de 0.2s
=> [builder 15/18] RUN NODE_ENV=production pnpm install --prod --no-opt 37.3s
=> [builder 16/18] RUN find . -type f -name "*.ts" -o -name "*.js.map" - 2.5s
=> [builder 17/18] RUN rm -rf packages/@n8n_io/eslint-config packages/ed 0.3s
=> [builder 18/18] RUN rm -rf patches .npmrc *.yaml node_modules/.cache 1.9s
=> [stage-1 2/5] COPY --from=builder /home/node /usr/local/lib/node_modu 4.1s
=> [stage-1 3/5] RUN ln -s /usr/local/lib/node_modules/n8n/packages/cli/ 0.3s
=> [stage-1 4/5] COPY docker/images/n8n-custom/docker-entrypoint.sh / 0.0s
=> [stage-1 5/5] RUN mkdir .n8n && chown node:node .n8n 0.2s
=> exporting to image 3.2s
=> => exporting layers 3.2s
=> => writing image sha256:b32c383a380c2d4b4e079f71d9579c911191cd5982bfe 0.0s
=> => naming to docker.io/library/n8n-custom 0.0s
Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
~/code/javascript/n8n-io-tmp/n8n | master ............... 5m 0s | 01:58:19 pm
> docker run -it --name test -p 9999:5678 n8n-custom
UserSettings were generated and saved to: /home/node/.n8n/config
Migrations in progress, please do NOT stop the process.
Initializing n8n process
Migrations finished.
n8n ready on 0.0.0.0, port 5678
Version: 0.217.2
Editor is now accessible via:
http://localhost:5678/
Press "o" to open in Browser.
Can you try the same start command and see if that works for you? It could just be a permission issue with the data volume.
taha-sonmez
commented
1 year ago Hi,
This is the first time for me to create a custom docker image for using external npm packages in n8n Code node. I'm having the same issue.
I didn't make any changes on Dockerfile. I've only installed an npm package.
Here are the logs of docker build:
taha.sonmez@TahaSonmez n8n % docker buildx build --platform linux/amd64 -t n8n-custom -f docker/images/n8n-custom/Dockerfile .
[+] Building 1451.9s (27/28)
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 1.84kB 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 35B 0.0s
=> [internal] load metadata for docker.io/n8nio/base:16 1451.7s
=> [auth] n8nio/base:pull token for registry-1.docker.io 0.0s
=> [internal] load build context 1.5s
=> => transferring context: 24.29MB 1.5s
=> CACHED [builder 1/18] FROM docker.io/n8nio/base:16@sha256:66deb688242abf0c3de148bede51fea89e42cce15e73f9300179e602c980ab8f 0.0s
=> [builder 2/18] COPY turbo.json package.json .npmrc pnpm-lock.yaml pnpm-workspace.yaml tsconfig.json ./ 0.2s
=> [builder 3/18] COPY scripts ./scripts 0.0s
=> [builder 4/18] COPY packages ./packages 0.8s
=> [builder 5/18] COPY patches ./patches 0.0s
=> [builder 6/18] RUN apk add --update libc6-compat jq 3.1s
=> [builder 7/18] RUN corepack enable && corepack prepare --activate 5.3s
=> [builder 8/18] RUN chown -R node:node . 4.4s
=> [builder 9/18] RUN pnpm install --frozen-lockfile 226.1s
=> [builder 10/18] RUN pnpm build 1109.6s
=> [builder 11/18] RUN rm -rf node_modules 7.7s
=> [builder 12/18] RUN jq 'del(.pnpm.patchedDependencies)' package.json > package.json.tmp; mv package.json.tmp package.json 0.4s
=> [builder 13/18] RUN jq '{name: .name, version: .version}' packages/editor-ui/package.json > editor-ui.tmp; mv editor-ui.tmp packages/editor-ui/package.json 0.3s
=> [builder 14/18] RUN jq '{name: .name, version: .version}' packages/design-system/package.json > design-system.tmp; mv design-system.tmp packages/design-system/package.json 0.3s
=> [builder 15/18] RUN NODE_ENV=production pnpm install --prod --no-optional 69.7s
=> [builder 16/18] RUN find . -type f -name "*.ts" -o -name "*.js.map" -o -name "*.vue" -o -name "tsconfig.json" -o -name "*.tsbuildinfo" | xargs rm -rf 3.5s
=> [builder 17/18] RUN rm -rf packages/@n8n_io/eslint-config packages/editor-ui/src packages/editor-ui/node_modules packages/design-system 0.3s
=> [builder 18/18] RUN rm -rf patches .npmrc *.yaml node_modules/.cache packages/**/node_modules/.cache packages/**/.turbo .config .cache .local .node /tmp/* 2.8s
=> [stage-1 2/5] COPY --from=builder /home/node /usr/local/lib/node_modules/n8n 6.5s
=> [stage-1 3/5] RUN ln -s /usr/local/lib/node_modules/n8n/packages/cli/bin/n8n /usr/local/bin/n8n 0.3s
=> [stage-1 4/5] COPY docker/images/n8n-custom/docker-entrypoint.sh / 0.0s
=> [stage-1 5/5] RUN mkdir .n8n && chown node:node .n8n 0.2s
=> exporting to image 2.9s
=> => exporting layers 2.9s
=> => writing image sha256:8f513a325176898f28fe6815556c8f8ebee0bebd2116439a57647d28e4c88e45 0.0s
=> => naming to docker.io/library/n8n-customAfter docker file is created, I've tagged and pushed it to my Docker hub account. Then, pull it in my server and run with below run command:
docker run -d --restart=always --log-opt max-size=10m --log-opt max-file=5 --platform linux/amd64 --name n8n-main -p 5678:5678 -v /root/n8n/.n8n:/home/node/.n8n --env-file=/root/n8n/main.env tahasnmz/n8n-custom:v0.1.2 n8n
Error is:
node:internal/modules/cjs/loader:1024
throw err;
^
Error: Cannot find module '/home/node/n8n'
at Function.Module._resolveFilename (node:internal/modules/cjs/loader:1021:15)
at Function.Module._load (node:internal/modules/cjs/loader:866:27)
at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
at node:internal/main/run_main_module:22:47 {
code: 'MODULE_NOT_FOUND',
requireStack: []
}Thank you for your help.
Joffcom
commented
1 year ago Hey @taha-sonmez
How are you adding the npm package?
Can you try the same command I ran above to see if that works for you? I have tried it from 3 machines with no issues so far.
taha-sonmez
commented
1 year ago Hi @Joffcom ,
Thank you for quick reply.
I used below commands on VS Code terminal to install external npm package:
pnpm install googleapis
pnpm install
pnpm build
I've used your command to run docker container, it worked.
UserSettings were generated and saved to: /home/node/.n8n/config
Migrations in progress, please do NOT stop the process.
Initializing n8n process
Migrations finished.
n8n ready on 0.0.0.0, port 5678
Version: 0.218.0
Editor is now accessible via:
http://localhost:5678/
Press "o" to open in Browser.But still I need these parameters:
docker run -d --restart=always --log-opt max-size=10m --log-opt max-file=5 --name n8n-main -p 5678:5678 -v /root/n8n/.n8n:/home/node/.n8n --env-file=/root/n8n/main.env tahasnmz/n8n-custom:v0.1.2 n8n
I couldn't understand what can be the reason. Also, here is my environment file for my n8n sandbox:
DATA_FOLDER=/root/n8n/
GENERIC_TIMEZONE=Europe/Istanbul
N8N_PAYLOAD_SIZE_MAX=160
NODE_FUNCTION_ALLOW_BUILTIN=*
NODE_FUNCTION_ALLOW_EXTERNAL=request,axios,form-data,googleapis
NODE_OPTIONS="--max-old-space-size=12360"Do you need the n8n option at the end 🤔
If you are just adding the google APIs package there is also an easier way you could just use the from option in a docker file to use our image then add a run to install the package.
taha-sonmez
commented
1 year ago I removed n8n option and it worked. I will use these image in my prod environment and environment is working in queue mode. So I will need 'n8n worker' option at the end eventually. Isn't it correct?
I couldn't understand the easier way, is there any community or github thread for that?
By the way, I've compared my building durations with yours while creating a Docker image, why does my build takes 5x longer? Any clue? (Chip: M1 Pro)
bracci-quest-it
commented
1 year ago Can you try the same start command and see if that works for you?
I tried on my local code to run the commnad:
pnpm startAnd the application runs as expected
Editor is now accessible via:
http://localhost:5678/
Press "o" to open in Browser.It could just be a permission issue with the data volume
How can i check if it is a volume permission problem? If so how can I resolve it?
taha-sonmez
commented
1 year ago Hi @bracci-quest-it ,
I believe @Joffcom meant running the docker container without "n8n":
docker run -it --name test -p 9999:5678 n8n-custom
It is working with that way but still I'm curious about what will happen when I want to start n8n container as a worker or webhook processor. I'm waiting Jon's response.
bracci-quest-it
commented
1 year ago Hi @taha-sonmez , okay i tried again to build my custom image:
n8n % docker build -t n8n-custom -f docker/images/n8n-custom/Dockerfile .
[+] Building 386.5s (27/27) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 37B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 35B 0.0s
=> [internal] load metadata for docker.io/n8nio/base:16 1.3s
=> [internal] load build context 0.4s
=> => transferring context: 391.53kB 0.4s
=> CACHED [builder 1/18] FROM docker.io/n8nio/base:16@sha256:66deb688242abf0c3de148bede51fea89e42cce15e73f9300179e602c980ab8 0.0s
=> CACHED [builder 2/18] COPY turbo.json package.json .npmrc pnpm-lock.yaml pnpm-workspace.yaml tsconfig.json ./ 0.0s
=> CACHED [builder 3/18] COPY scripts ./scripts 0.0s
=> [builder 4/18] COPY packages ./packages 1.0s
=> [builder 5/18] COPY patches ./patches 0.0s
=> [builder 6/18] RUN apk add --update libc6-compat jq 1.1s
=> [builder 7/18] RUN corepack enable && corepack prepare --activate 3.0s
=> [builder 8/18] RUN chown -R node:node . 3.6s
=> [builder 9/18] RUN pnpm install --frozen-lockfile 44.5s
=> [builder 10/18] RUN pnpm build 270.2s
=> [builder 11/18] RUN rm -rf node_modules 6.3s
=> [builder 12/18] RUN jq 'del(.pnpm.patchedDependencies)' package.json > package.json.tmp; mv package.json.tmp package.json 0.3s
=> [builder 13/18] RUN jq '{name: .name, version: .version}' packages/editor-ui/package.json > editor-ui.tmp; mv editor-ui.tm 0.2s
=> [builder 14/18] RUN jq '{name: .name, version: .version}' packages/design-system/package.json > design-system.tmp; mv desi 0.2s
=> [builder 15/18] RUN NODE_ENV=production pnpm install --prod --no-optional 38.3s
=> [builder 16/18] RUN find . -type f -name "*.ts" -o -name "*.js.map" -o -name "*.vue" -o -name "tsconfig.json" -o -name "*. 2.1s
=> [builder 17/18] RUN rm -rf packages/@n8n_io/eslint-config packages/editor-ui/src packages/editor-ui/node_modules packages/ 0.2s
=> [builder 18/18] RUN rm -rf patches .npmrc *.yaml node_modules/.cache packages/**/node_modules/.cache packages/**/.turbo .c 1.9s
=> [stage-1 2/5] COPY --from=builder /home/node /usr/local/lib/node_modules/n8n 4.8s
=> [stage-1 3/5] RUN ln -s /usr/local/lib/node_modules/n8n/packages/cli/bin/n8n /usr/local/bin/n8n 0.3s
=> [stage-1 4/5] COPY docker/images/n8n-custom/docker-entrypoint.sh / 0.0s
=> [stage-1 5/5] RUN mkdir .n8n && chown node:node .n8n 0.2s
=> exporting to image 2.8s
=> => exporting layers 2.8s
=> => writing image sha256:1fbb53ca0e0ac599d2278e96f28bbd8fb0a5835fcf1c72d21413aa95447565b2 0.0s
=> => naming to docker.io/library/n8n-custom 0.0s
Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix themAfter that i run the command that you mentioned:
docker run -it --name test -p 9999:5678 n8n-customand the app still crashed without any information, with the following output:
Initializing n8n process
Migrations in progress, please do NOT stop the process.
UserSettings were generated and saved to: /home/node/.n8n/config
Migrations finished.I even tried with my .env file the output was:
2023-03-06T09:50:31.473Z | info | Initializing n8n process "{ file: 'start.js', function: 'run' }"
UserSettings were generated and saved to: /home/node/.n8n/config
2023-03-06T09:50:32.625Z | debug | Lazy Loading credentials and nodes from n8n-nodes-base "{\n credentials: 312,\n nodes: 402,\n file: 'DirectoryLoader.js',\n function: 'loadAll'\n}"Hey,
To answer a quick question... I am using an M1 Macbook pro as well so I am not sure why it is quicker for me but I have been using a clean n8n repo with no changes at all just to make sure it works. In theory worker should be ok but some more testing might be needed.
The easier way if you are just adding an npm package would be...
FROM n8nio/n8n:latest
RUN npm install -g PACKAGE-NAMEIt looks like the images are building which is great so it will just be a case of working out if there is something local preventing it from running or something else.
@bracci-quest-it have you tried on a fresh download of the n8n code without making any changes just to rule out anything odd there?
taha-sonmez
commented
1 year ago Hi again @Joffcom,
I've just created a Docker file as you suggested, and it worked like a charm. Also, I've tried to create a Docker image from /docker/images/n8n/Dockerfile file with RUN npm install -g googleapis addition and it worked as well with "n8n" option too. Thank you for your help!
One more Docker-newbie question 😄
Default n8n image size is 582 MB Size of an image that created from n8n-custom/Dockerfile is 445 MB And size of my current image which I created with your suggested Docker file is 737 MB
What can be the reason behind that?
Joffcom
commented
1 year ago Hey @taha-sonmez,
I think it all comes down to some compression options, I have not worked out that side of docker yet though but it could be where the googleapis package has a bunch of extra bits to it but I am not sure.
bracci-quest-it
commented
1 year ago Hey, I found the problem, and it was in one of my custom nodes. In the icon property, I referenced the .svg icon with a different name from the real one, there was an upper letter instead of a lower letter. Before close this issue i would like to ask you something, I really love this project and i'd like to know if exists an official date for the 1.0 version.
Joffcom
commented
1 year ago Hey @bracci-quest-it,
No date yet for the 1.0, I have not even seen a rough internal date for it yet but I know we have started planning it.
Joffcom
commented
1 year ago Hey @bracci-quest-it,
A v1.0 preview will be live very soon (maybe a week or so). I am going to mark this issue as closed for now.
Describe the bug
After create a n8n-custom image with the following command:
I create an .env file with the following variables:
After that, using the n8n docker documentation (https://docs.n8n.io/hosting/installation/docker/#prerequisites) I start the container with the following command:
And I obtained the following output:
I tried to start the container without the n8n-start command, the container didn't run and the only output was:
For the sake of completeness i post the previous command:
Important to notice that the app in local with the use of the command
runs fine.
Expected behavior The container runs fine.
Environment: