wurstmeister / kafka-docker

Dockerfile for Apache Kafka
http://wurstmeister.github.io/kafka-docker/
Apache License 2.0
6.9k stars 2.73k forks source link

Update openjdk image #659

Closed mangas closed 3 years ago

mangas commented 3 years ago

The previous image was crashing on M1. After trying a few different images, this one seems to solve the problem

Closes https://github.com/wurstmeister/kafka-docker/issues/647

mangas commented 3 years ago

@huang-x-h could you please review this since you seem to have been the original author of this code

m3co-code commented 3 years ago

Can confirm, this works also in my setup on the M1 👍 Thanks!

huang-x-h commented 3 years ago

@huang-x-h could you please review this since you seem to have been the original author

Sorry, the author is @wurstmeister , i just upgrade the kafka version to 2.7.0

wurstmeister commented 3 years ago

Hi,

this change breaks the build.

The problem seems to be a change in behaviour of curl's -I flag when there is a 404

old version of curl

/ # curl -f -I -s https://ftp.wayne.edu/apache/kafka/2.6.0/kafka_2.13-2.6.0.tgz
/ # 

new version of curl

/ # curl -f -I -s https://ftp.wayne.edu/apache/kafka/2.6.0/kafka_2.13-2.6.0.tgz
HTTP/1.1 404 Not Found
Date: Sat, 22 May 2021 06:56:48 GMT
Server: Apache/2.4.6 () OpenSSL/1.0.2k-fips
Content-Type: text/html; charset=iso-8859-1

This results in the following error during build:

sh: 404: unknown operand

according to https://curl.se/mail/tracker-2013-05/0088.html it's expected that the header is printed.

I did a quick test and it seems if we use a range we can achieve the same test:

/ # curl -f -s -r 0-1 https://ftp.wayne.edu/apache/kafka/2.6.0/kafka_2.13-2.6.0.tgz
/ # echo $?
22
/ # curl -f -s -r 0-1 https://ftp.wayne.edu/apache/kafka/2.8.0/kafka_2.13-2.8.0.tgz
/ # echo $?
0
mangas commented 3 years ago

I'm not sure what the previous comment is referring to, is it a build for a previous version of kafka?

 docker build --no-cache  .
[+] Building 29.0s (9/9) FINISHED
 => [internal] load build definition from Dockerfile                                                                                                                                                            0.0s
 => => transferring dockerfile: 37B                                                                                                                                                                             0.0s
 => [internal] load .dockerignore                                                                                                                                                                               0.0s
 => => transferring context: 2B                                                                                                                                                                                 0.0s
 => [internal] load metadata for docker.io/azul/zulu-openjdk-alpine:8u292-8.54.0.21                                                                                                                             0.6s
 => CACHED [1/4] FROM docker.io/azul/zulu-openjdk-alpine:8u292-8.54.0.21@sha256:86424b902f10c1e8f1484e6e2b79f2c9e0eaf0be9bbdf8f4933e49b5f2521980                                                                0.0s
 => [internal] load build context                                                                                                                                                                               0.0s
 => => transferring context: 243B                                                                                                                                                                               0.0s
 => [2/4] COPY download-kafka.sh start-kafka.sh broker-list.sh create-topics.sh versions.sh /tmp/                                                                                                               0.1s
 => [3/4] RUN apk add --no-cache bash curl jq docker  && chmod a+x /tmp/*.sh  && mv /tmp/start-kafka.sh /tmp/broker-list.sh /tmp/create-topics.sh /tmp/versions.sh /usr/bin  && sync && /tmp/download-kafka.s  27.5s
 => [4/4] COPY overrides /opt/overrides                                                                                                                                                                         0.0s
 => exporting to image                                                                                                                                                                                          0.7s
 => => exporting layers                                                                                                                                                                                         0.7s
 => => writing image sha256:98e803fd50b188bdf31d360790d89deefedb8a1ce8b8fbc3189ed4ebc3880799
mangas commented 3 years ago

Also there seems to be a rate limit problem with the build at the moment "Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit"

The change seems to be passing on a few and the rest gets stuck with the rate limit but this should then fix the issue?

wurstmeister commented 3 years ago

Thanks for updating the PR @mangas, could you please also update the changelog?

regarding the rate limit, I need to check what we can do to address that.

mangas commented 3 years ago

@wurstmeister updated the changelog as well. Did you have a chance to look into the rate limit issue? Do you think we could merge this in?

wurstmeister commented 3 years ago

@mangas thanks for the update. I've made a couple changes to reduce the chance of the rate limit issue happening, at the very least when building the master branch. could you please update your branch with the latest changes, then we can merge your change.

mangas commented 3 years ago

@wurstmeister seems like we're still being rate limited but I've rebased as requested

mangas commented 3 years ago

Perhaps making the build steps sequencial could help with the rate limiting problem?