Unable to rebuild docker image due to 403 on hbase download and opentsdb compile errors

[itzg]

I tried to rebuild the docker image as of https://github.com/GoogleCloudPlatform/opentsdb-bigtable/commit/00ceae08c2359ab599a3aedbe015a4294a48c01b by running docker build -t opentsdb-bigtable . in the build directory, but it failed with:

Processing triggers for mime-support (3.62) ...
Processing triggers for libgdk-pixbuf2.0-0:amd64 (2.38.1+dfsg-1) ...
  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0
curl: (22) The requested URL returned error: 403 Forbidden

gzip: stdin: unexpected end of file
tar: Child returned status 1
tar: Error is not recoverable: exiting now
The command '/bin/sh -c apt-get update && apt-get install -y autoconf make unzip gnuplot curl git &&     curl -f http://storage.googleapis.com/cloud-bigtable/hbase-dist/hbase-1.2.1/hbase-1.2.1-bin.tar.gz | tar zxf -  &&     mkdir -p hbase-1.2.1/lib/bigtable &&     curl http://repo1.maven.org/maven2/com/google/cloud/bigtable/bigtable-hbase-1.2/0.9.5.1/bigtable-hbase-1.2-0.9.5.1.jar       -f -o hbase-1.2.1/lib/bigtable/bigtable-hbase-1.2-0.9.5.1.jar &&     curl http://repo1.maven.org/maven2/io/netty/netty-tcnative-boringssl-static/1.1.33.Fork19/netty-tcnative-boringssl-static-1.1.33.Fork19.jar       -f -o hbase-1.2.1/lib/netty-tcnative-boringssl-static-1.1.33.Fork19.jar &&     echo 'export HBASE_CLASSPATH="$HBASE_HOME/lib/bigtable/bigtable-hbase-1.2-0.9.5.1.jar:$HBASE_HOME/lib/netty-tcnative-boringssl-static-1.1.33.Fork19.jar"' >> /hbase-1.2.1/conf/hbase-env.sh &&     echo 'export HBASE_OPTS="${HBASE_OPTS} -Xms1024m -Xmx2048m"' >> /hbase-1.2.1/conf/hbase-env.sh' returned a non-zero code: 2

I locally modified the Dockerfile and switched to an apache archive location: https://archive.apache.org/dist/hbase/1.2.1/hbase-1.2.1-bin.tar.gz. It got past the hbase download but failed on the http://repo1.maven.org/ access since HTTPS is required as of Jan 15, 2020.

curl: (22) The requested URL returned error: 501 HTTPS Required
The command '/bin/sh -c apt-get update && apt-get install -y autoconf make unzip gnuplot curl git &&     curl -f https://archive.apache.org/dist/hbase/1.2.1/hbase-1.2.1-bin.tar.gz | tar zxf -  &&     mkdir -p hbase-1.2.1/lib/bigtable &&     curl http://repo1.maven.org/maven2/com/google/cloud/bigtable/bigtable-hbase-1.2/0.9.5.1/bigtable-hbase-1.2-0.9.5.1.jar       -f -o hbase-1.2.1/lib/bigtable/bigtable-hbase-1.2-0.9.5.1.jar &&     curl http://repo1.maven.org/maven2/io/netty/netty-tcnative-boringssl-static/1.1.33.Fork19/netty-tcnative-boringssl-static-1.1.33.Fork19.jar       -f -o hbase-1.2.1/lib/netty-tcnative-boringssl-static-1.1.33.Fork19.jar &&     echo 'export HBASE_CLASSPATH="$HBASE_HOME/lib/bigtable/bigtable-hbase-1.2-0.9.5.1.jar:$HBASE_HOME/lib/netty-tcnative-boringssl-static-1.1.33.Fork19.jar"' >> /hbase-1.2.1/conf/hbase-env.sh &&     echo 'export HBASE_OPTS="${HBASE_OPTS} -Xms1024m -Xmx2048m"' >> /hbase-1.2.1/conf/hbase-env.sh' returned a non-zero code: 22

I changed http://repo1.maven.org/ to https://repo1.maven.org/ and it made it to the opentsdb-bigtable build step, but encountered a compilation error which is a separate I will report against their repo as of https://github.com/OpenTSDB/opentsdb/commit/12bc21ddcc3d559c8d1ca696906888a88d056876

[itzg]

BTW, I had to rebuild my own image because the v1 image in the registry is very old and I needed 2.4 features, such as the rollup API.

[varundhussa]

Added image v2 which has v2.4 and the new Bigtable connector