Open gogagum opened 1 year ago
Hey!
Got the same problem, the base image cannot be built using make build
.
As a workaround, I did some changes in the Makefile and pulled the base image manually. Here's what I did:
In Makefile
from master
:
current_branch := 2.0.0-hadoop3.2.1-java8
#docker build -t bde2020/hadoop-base:$(current_branch) ./base
Then, run make build
. It should now build images with the appropriate tag.
Finally, run the following docker command to pull a working base image:
$ docker pull bde2020/hadoop-base:2.0.0-hadoop3.2.1-java8
That's because debian 9 (stretch) is not maintained anymore. You can update the image to debian:bullseye-slim
instead.
You also need to use openjdk-11 instead since openjdk-8 is not in the debian packages list anymore.
Additonally, you need to use https://archive.apache.org
for the hadoop URL. (Versions available here)
All in all, here are the changes:
FROM debian:bullseye-slim
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
openjdk-11-jdk \
net-tools \
curl \
netcat \
gnupg \
libsnappy-dev \
&& rm -rf /var/lib/apt/lists/*
ENV JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64/
...
ENV HADOOP_URL https://archive.apache.org/dist/hadoop/common/hadoop-$HADOOP_VERSION/hadoop-$HADOOP_VERSION.tar.gz
Anyhow, I did fork this repo since I need it for a small lab demo. The setup is more streamlined there and is as easy as running
# Build the cluster
make build
# Run the cluster
make up
# Get a shell into the hadoop cluster
make shell
When I try building base hadoop image, I get errors from
apt
in debian:Base image needs to be changed from
debian
, or something should be done with debian repositories.