search

jitsucom / jitsu

Jitsu is an open-source Segment alternative. Fully-scriptable data ingestion engine for modern data teams. Set-up a real-time data pipeline in minutes, not days
https://jitsu.com
MIT License
4.13k stars 296 forks source link

Cannot build Docker images - build process freezes #1145

Closed johnpitchko closed 2 weeks ago

johnpitchko commented 2 weeks ago

Summary

The Docker image build appears to freeze/stall during building.

The command I am running to build the images: docker buildx build --platform linux/amd64 . -f all.Dockerfile

The build process starts, but freezes during the RUN pnpm build step. As you can see in the screenshot, the build has been running for >1300 seconds (20 minutes).

Screenshot 2024-11-14 at 13 54 44

System configuration and versions

Using Jitsu Next on latest main branch. Docker version 27.2.0.

johnpitchko commented 2 weeks ago

Curiously, dropping the --platform linux/amd64 argument allowed the build to succeed:

Screenshot 2024-11-14 at 14 08 50
vklimontovich commented 2 weeks ago

Are you building this on Apple Silicon? They have a pretty poor simulation layer for arm64

johnpitchko commented 2 weeks ago

👋 yes I am on a Macbook Pro M1. If that is the case, I can look into other solutions to build the image. Are you on Apple Silicon and if so, what do you use to build?

vklimontovich commented 2 weeks ago

For local development, just drop --platform option, and it will build a native arm64 image. If you want to deploy it somewhere, I believe it's going to be amd64, so the best way to do so is to get a AMD64/Linux instance from any cloud provider. That's what we use in our CI/CD workflow

Another approach would be using docker build cloud. We have a plans to switch to it and add a support to build scripts, but we are not there yet. But you can try hack the script and maybe make a PR

johnpitchko commented 2 weeks ago

That's cool thank you. Also, pardon my Docker newbie question, but I noticed that only a single image is put from from that script. I assume that image is for the Console. How do I build images for Ingest and Rotor and Bulker (but I believe that is in a separate repo)?

PS is your documentation in a public repo? I would be happy to contribute to it as I tinker with the self-hosted version.

vklimontovich commented 2 weeks ago

It's a multi-stage docker file, meaning it can produce several images. It's controlled by --target option. I don't believe we use ./consolebuild.sh anywhere. Look at pnpm run build-scripts, and ./release.sh.

vklimontovich commented 2 weeks ago

Closing this one since nothing should be done here

johnpitchko commented 1 week ago

Agreed - thank you for your help @vklimontovich