paypal / junodb

JunoDB is PayPal's home-grown secure, consistent and highly available key-value store providing low, single digit millisecond, latency at any scale.
Apache License 2.0
2.56k stars 163 forks source link

Unable to build junoDB using docker #140

Open viralnatani opened 1 year ago

viralnatani commented 1 year ago

I was following the steps mentioned in the README.md file to build junoDB using docker. But the build.sh threw an error:

Screenshot 2023-06-05 at 5 52 06 PM

This error originates from Makefile located at path ~/docker/Makefile at line number 58 while creating a build for junoclustercfg.

Upon further investigation, it looks like some dependency folder is missing.

  1. Build command uses dockerfile located at ~/docker/cluster-service-build/Dockerfile.

    image
  2. For building a docker image for junoclustercfg it copies files from /juno/bin folder from juno-build. As shown below

    image

But when juno-build is investigated it turns out there is no folder bin located at /juno

  1. This can be traced back to Dockerfile (located at ~/docker/build/Docker)used to build juno-build . It copies /src/juno to /juno folder. So /src/juno must contains bin folder. But it isn't there. image
NeetishPathak commented 1 year ago

juno docker build is a multi-stage build. juno-dev and juno-build image must be created before other images like junoserv,junostoragserv,junoclustercfg,junoclusterserv etc

from the junodb home folder , follow the steps

This will generate juno-build image that will be used to copy juno binaries. /juno/bin directory is where binaries are located , it is configured in the golang install command https://github.com/paypal/junodb/blob/dev/docker/build/build.sh#L50

This is followed by building individual images for junodb services you should have the following images created eventually REPOSITORY TAG
ghcr.io/paypal/junodb/junoclient:latest ghcr.io/paypal/junodb/junostorageserv:latest ghcr.io/paypal/junodb/junoserv:latest ghcr.io/paypal/junodb/junoclustercfg:latest ghcr.io/paypal/junodb/junoclusterserv:latest ghcr.io/paypal/junodb/juno-build:latest ghcr.io/paypal/junodb/juno-dev:latest

It is likely that juno-build image is not successfully built in this case You may see a demo here - https://github.com/paypal/junodb/blob/dev/docs/videos.md

Can you mention the environment you are using. docker version, os etc. I am not able to reproduce this issue

viralnatani commented 1 year ago

I am using macOS and the docker version is 20.10.11.

MichelBoucey commented 1 year ago

FYI build process is flawless on an up-to-date ArchLinux running docker version 24.0.2.

NeetishPathak commented 1 year ago

If the juno-build image is not correctly built , it will not have the required binaries such as /opt/juno/junoctl etc.

When building macOs with docker desktop , I saw the issue when corporate firewall MITMing your https connections and some go modules certs could not be verified. As a result, juno-build step was not successful and you'll need an appropriate CA . The similar issue was reported on golang project https://github.com/golang/go/issues/46428 , https://github.com/golang/go/issues/45569

I have added the juno-build image script to exit during juno-build if certs verification fail. You can try to re-build and confirm if that was the issue.

Otherwise, building on macOS on github actions workflow is working fine. https://github.com/paypal/junodb/actions/runs/5259960159/jobs/9513643692