Closed mangas closed 3 years ago
@huang-x-h could you please review this since you seem to have been the original author of this code
Can confirm, this works also in my setup on the M1 👍 Thanks!
@huang-x-h could you please review this since you seem to have been the original author
Sorry, the author is @wurstmeister , i just upgrade the kafka version to 2.7.0
Hi,
this change breaks the build.
The problem seems to be a change in behaviour of curl's -I
flag when there is a 404
old version of curl
/ # curl -f -I -s https://ftp.wayne.edu/apache/kafka/2.6.0/kafka_2.13-2.6.0.tgz
/ #
new version of curl
/ # curl -f -I -s https://ftp.wayne.edu/apache/kafka/2.6.0/kafka_2.13-2.6.0.tgz
HTTP/1.1 404 Not Found
Date: Sat, 22 May 2021 06:56:48 GMT
Server: Apache/2.4.6 () OpenSSL/1.0.2k-fips
Content-Type: text/html; charset=iso-8859-1
This results in the following error during build:
sh: 404: unknown operand
according to https://curl.se/mail/tracker-2013-05/0088.html it's expected that the header is printed.
I did a quick test and it seems if we use a range we can achieve the same test:
/ # curl -f -s -r 0-1 https://ftp.wayne.edu/apache/kafka/2.6.0/kafka_2.13-2.6.0.tgz
/ # echo $?
22
/ # curl -f -s -r 0-1 https://ftp.wayne.edu/apache/kafka/2.8.0/kafka_2.13-2.8.0.tgz
/ # echo $?
0
I'm not sure what the previous comment is referring to, is it a build for a previous version of kafka?
docker build --no-cache .
[+] Building 29.0s (9/9) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 37B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/azul/zulu-openjdk-alpine:8u292-8.54.0.21 0.6s
=> CACHED [1/4] FROM docker.io/azul/zulu-openjdk-alpine:8u292-8.54.0.21@sha256:86424b902f10c1e8f1484e6e2b79f2c9e0eaf0be9bbdf8f4933e49b5f2521980 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 243B 0.0s
=> [2/4] COPY download-kafka.sh start-kafka.sh broker-list.sh create-topics.sh versions.sh /tmp/ 0.1s
=> [3/4] RUN apk add --no-cache bash curl jq docker && chmod a+x /tmp/*.sh && mv /tmp/start-kafka.sh /tmp/broker-list.sh /tmp/create-topics.sh /tmp/versions.sh /usr/bin && sync && /tmp/download-kafka.s 27.5s
=> [4/4] COPY overrides /opt/overrides 0.0s
=> exporting to image 0.7s
=> => exporting layers 0.7s
=> => writing image sha256:98e803fd50b188bdf31d360790d89deefedb8a1ce8b8fbc3189ed4ebc3880799
Also there seems to be a rate limit problem with the build at the moment "Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit"
The change seems to be passing on a few and the rest gets stuck with the rate limit but this should then fix the issue?
Thanks for updating the PR @mangas, could you please also update the changelog?
regarding the rate limit, I need to check what we can do to address that.
@wurstmeister updated the changelog as well. Did you have a chance to look into the rate limit issue? Do you think we could merge this in?
@mangas thanks for the update. I've made a couple changes to reduce the chance of the rate limit issue happening, at the very least when building the master branch. could you please update your branch with the latest changes, then we can merge your change.
@wurstmeister seems like we're still being rate limited but I've rebased as requested
Perhaps making the build steps sequencial could help with the rate limiting problem?
The previous image was crashing on M1. After trying a few different images, this one seems to solve the problem
Closes https://github.com/wurstmeister/kafka-docker/issues/647