ionutpopa commented 1 month ago

arm64) on docker build

Description

Added the following platforms in the build file from the hooks folder: linux/amd64 and linux/arm64

Documentation

Now Apache Drill Docker image is available for both linux/arm64 and linux/amd64

Testing

Built images locally

cgivre commented 1 month ago

@ionutpopa Thanks for this. Forgive the question as I'm not a Docker expert but do we need to add windows/amd64, and basically all other platforms that Drill should run on?

ionutpopa commented 1 month ago

@cgivre IMHO we could keep only the linux platforms as the original docker works on Windows just fine. But if we think about Drill as a software that we want to run on all platforms, maybe we need to add windows/amd64. What do you think?

cgivre commented 1 month ago

@ionutpopa People do run Drill on Windows.... As long as it works for them, I'm fine with keeping it as is. @jnturton Do you have an opinion on this?

jnturton commented 1 month ago

The Linux container images will work for Windows users through WSL or Hyper-V which is how they must already be running most of their conatiners. Actually building on a Windows container base image would require deeper changes from us, I think, and someone would have to check the license to see whether we'd be free to redistribute the Microsoft binaries included as an Apache project.

cgivre commented 1 month ago

The Linux container images will work for Windows users through WSL or Hyper-V which is how they must already be running most of their conatiners. Actually building on a Windows container base image would require deeper changes from us, I think, and someone would have to check the license to see whether we'd be free to redistribute the Microsoft binaries included as an Apache project.

In that case, LGTM +1

jnturton commented 1 month ago

Marked as draft until we've triggered a test in Docker Hub (which I'll do).

ionutpopa commented 1 month ago

@jnturton The ci/dockercloud-stage failed and when checking the details I saw {"error": "Our service is temporarily unavailable. We'll be back soon!"}, I guess that's fine?

jnturton commented 1 month ago

@ionutpopa, below is what happened in Docker Hub.

2024-10-14T06:58:22Z Executing build hook...
2024-10-14T06:58:29Z ERROR: multiple platforms feature is currently not supported for docker driver. Please switch to a different driver (eg. "docker buildx create --use")

We can read up on and try docker buildx. I have only skimmed the Internet but it sounds like it might resemble

# docker buildx create --use
docker buildx build --platform linux/amd64,linux/arm64

EDIT: I've commented out the docker buildx create command. That's meant to a once-off setup, and there's a chance it's already been done in Docker Hub and we don't need to run it at all.

jnturton commented 1 month ago

I got a build going after putting docker buildx create --use back in. It appeared to make progress on two parallel builds but it got killed after running for 4 hours (see the final line below). We'll probably have to split things up differently and, given that Docker Hub access is required to see what does or doesn't work, I'll probably have to carry on with that in the 8260-arm64-image branch.

2024-10-18T12:06:49Z #21 1435.4 [output clipped, log limit 2MiB reached]
2024-10-18T12:15:35Z #21 ...
2024-10-18T12:15:35Z 
2024-10-18T12:15:35Z #20 [linux/amd64 build 4/5] RUN mvn clean install -DskipTests
2024-10-18T12:15:35Z #20 DONE 1961.4s
2024-10-18T12:15:35Z 
2024-10-18T12:15:35Z #22 [linux/amd64 build 5/5] RUN VERSION=$(mvn -q -Dexec.executable=echo -Dexec.args='${project.version}' --non-recursive exec:exec)  && mkdir /opt/drill  && mv distribution/target/apache-drill-${VERSION}/apache-drill-${VERSION}/* /opt/drill  && chmod -R +r /opt/drill
2024-10-18T12:15:56Z #22 DONE 20.9s
2024-10-18T12:15:56Z 
2024-10-18T12:15:56Z #21 [linux/arm64 build 4/5] RUN mvn clean install -DskipTests
2024-10-18T12:15:57Z #21 ...
2024-10-18T12:15:57Z 
2024-10-18T12:15:57Z #23 [linux/amd64 stage-1 3/3] COPY --from=build /opt/drill /opt/drill
2024-10-18T12:16:01Z #23 DONE 3.6s
2024-10-18T12:16:01Z 
2024-10-18T12:16:01Z #21 [linux/arm64 build 4/5] RUN mvn clean install -DskipTests
2024-10-18T12:48:56Z #21 3962.5 Warning: Lookahead adequacy checking not being performed since option LOOKAHEAD is more than 1.  Set option FORCE_LA_CHECK to true to force checking.
2024-10-18T13:03:45Z The build was cancelled or exceeded the maximum execution time.

apache / drill