automatic build attempts to pull huge docker images twice (and fails at the moment)

[pleasantone]

First off, thank you so much for developing the automatic build system. I know it's a work in progress, and I'm happy to help with feedback if it's useful to you. Since I'm starting this from a different state that you are, I may be running into things you've already worked around.

I started with a completely blank-slate machine, docker and python et al installed, and a fresh clone of master, running automatic build for the very first time, so I have no locally cached images.

Working off of master as of today, and starting with a blank slate, automatic-build first tries pulls the :latest docker image (24gb) and then attempts to pull the nw49 tagged docker image. Yesterday, I believe pulling :nw49 worked, today, starting with a blank build machine, it didn't succeed at pulling that image.

So two issues:

1. The size of these docker images its huge, so we probably shouldn't be pulling what is almost the same image twice when starting up.
2. manifest for laslaul/nwjs-arm-build-env:nw49 not found

Output:

pst@ip-172-30-4-90:~$ git clone https://github.com/LeonardLaszlo/nw.js-armv7-binaries.git Cloning into 'nw.js-armv7-binaries'... remote: Enumerating objects: 265, done. remote: Counting objects: 100% (265/265), done. remote: Compressing objects: 100% (161/161), done. remote: Total 555 (delta 188), reused 178 (delta 102), pack-reused 290 Receiving objects: 100% (555/555), 130.21 KiB | 10.02 MiB/s, done. Resolving deltas: 100% (315/315), done.

pst@ip-172-30-4-90:~$ cd nw.js-armv7-binaries/ pst@ip-172-30-4-90:~/nw.js-armv7-binaries$ bash automatic-build.sh >& BUILD1_LOG & [1] 22871

pst@ip-172-30-4-90:~/nw.js-armv7-binaries$ tail -100f BUILD1_LOG 2020-10-22 15:43:17 TODO install curl, python, docker if needed 2020-10-22 15:43:17 Start parsing parameters 2020-10-22 15:43:17 Set positional arguments in their place 2020-10-22 15:43:17 Done parsing parameters 2020-10-22 15:43:17 Get the active branch from the official repo 2020-10-22 15:43:17 NW.js active branch: nw49 2020-10-22 15:43:17 Docker repository: laslaul/nwjs-arm-build-env 2020-10-22 15:43:17 Docker parameters: -H unix:///var/run/docker.sock 2020-10-22 15:43:17 Check whether the image exists on the docker host 2020-10-22 15:43:17 The laslaul/nwjs-arm-build-env image does not exist on the docker host. Pulling: laslaul/nwjs-arm-build-env Using default tag: latest latest: Pulling from laslaul/nwjs-arm-build-env 171857c49d0f: Pulling fs layer 419640447d26: Pulling fs layer 61e52f862619: Pulling fs layer 4c81915866aa: Pulling fs layer 3f8fd017cc1c: Pulling fs layer 2b6e377f3a5a: Pulling fs layer 4c81915866aa: Waiting 3f8fd017cc1c: Waiting 2b6e377f3a5a: Waiting 61e52f862619: Download complete 419640447d26: Verifying Checksum 419640447d26: Download complete 3f8fd017cc1c: Verifying Checksum 3f8fd017cc1c: Download complete 4c81915866aa: Verifying Checksum 4c81915866aa: Download complete 171857c49d0f: Verifying Checksum 171857c49d0f: Download complete 171857c49d0f: Pull complete 419640447d26: Pull complete 61e52f862619: Pull complete 4c81915866aa: Pull complete 3f8fd017cc1c: Pull complete 2b6e377f3a5a: Verifying Checksum 2b6e377f3a5a: Download complete 2b6e377f3a5a: Pull complete Digest: sha256:d9ed0fcc6eb5ea693bc069aab09deb43d80ca5b7d29468f2f77005ff17801cea Status: Downloaded newer image for laslaul/nwjs-arm-build-env:latest docker.io/laslaul/nwjs-arm-build-env:latest 2020-10-22 15:56:51 Found image of laslaul/nwjs-arm-build-env on dockerhub 2020-10-22 15:56:51 Start a container from the laslaul/nwjs-arm-build-env image Unable to find image 'laslaul/nwjs-arm-build-env:nw49' locally docker: Error response from daemon: manifest for laslaul/nwjs-arm-build-env:nw49 not found: manifest unknown: manifest unknown. See 'docker run --help'.

[LeonardLaszlo]

Hi there, thanks for your feedback.

The latest stable build is nw48, also tagged with the latest tag. Because I am still testing the script and experimenting I've pushed the nw49 image without thinking about the implications, and as it turned out is unstable.

I'll delete it so it won't create any confusion.

Using the nw48 docker image (marked with latest) you can build older versions, they are also stable. (From nw41-42 to nw48).

Under the releases tab, there is no release for nw49 available yet. That is because I cannot build nw49, I am getting an error during the build. See https://github.com/nwjs/nw.js/issues/7603

You can build other versions on nw on the docker image marked with nw48. Most probably there won't be another docker image, because thanks to git once you get a repo it is easy to switch between the branches and we don't have to pull the whole codebase. However, when you switch to another branch the following repositories will be cloned every time:

getGitRepository "https://github.com/nwjs/nw.js" "$NWJSDIR/src/content/nw"
getGitRepository "https://github.com/nwjs/node" "$NWJSDIR/src/third_party/node-nw"
getGitRepository "https://github.com/nwjs/v8" "$NWJSDIR/src/v8"

That takes 11 minutes currently with a 20 Mbps internet connection, so most probably it won't be a big deal.

Thanks again.

[LeonardLaszlo]

Probably I've already deleted nw49 from docker hub. I cannot find it.

[LeonardLaszlo]

I think this issue is fixed with the latest version. I am closing it because it's not relevant anymore.

[pleasantone]

Ok I have a build in progress and have made it past that point. Thank you. I was waiting to reply until it was complete.

For the record I had to use -b nw48 On Oct 25, 2020, 1:21 AM -0700, Leonard László notifications@github.com, wrote:

Closed #41. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[LeonardLaszlo]

Fyi you can also build nw49 with the latest version. I have added a patch to fix a failing assertion.