jwz16
commented
1 year ago Successfully built Hadoop environment by docker.
Artifact
sophie@sophie-ubuntu:~/bu/cs528/hadoop$ docker image ls | grep hadoop
hadoop-build-1000 latest 1b3d8fc2cd14 10 minutes ago 2.42GB
hadoop-build latest f443952733ce 10 minutes ago 2.42GBThe docker image hadoop-build-1000 contains the required environment for building Hadoop.
Build logs (partial)
sophie@sophie-ubuntu:~/bu/cs528/hadoop$ ./start-build-env.sh
[+] Building 580.0s (22/22) FINISHED docker:default
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 3.14kB 0.0s
=> [internal] load metadata for docker.io/library/ubuntu:focal 0.7s
=> [ 1/17] FROM docker.io/library/ubuntu:focal@sha256:33a5cc25d22c45900796a1aca487ad7a7cb09f09ea00b779e3b2026b4fc2faba 1.7s
=> => resolve docker.io/library/ubuntu:focal@sha256:33a5cc25d22c45900796a1aca487ad7a7cb09f09ea00b779e3b2026b4fc2faba 0.0s
=> => sha256:6df89402372646d400cf092016c28066391a26f5d46c00b1153e75003465484d 2.30kB / 2.30kB 0.0s
=> => sha256:edaedc954fb53f42a7754a6e2d1b57f091bc9b11063bc445c2e325ea448f8f68 27.51MB / 27.51MB 0.6s
=> => sha256:33a5cc25d22c45900796a1aca487ad7a7cb09f09ea00b779e3b2026b4fc2faba 1.13kB / 1.13kB 0.0s
=> => sha256:3246518d9735254519e1b2ff35f95686e4a5011c90c85344c1f38df7bae9dd37 424B / 424B 0.0s
=> => extracting sha256:edaedc954fb53f42a7754a6e2d1b57f091bc9b11063bc445c2e325ea448f8f68 1.0s
=> [internal] load build context 0.0s
=> => transferring context: 42.99kB 0.0s
=> [ 2/17] WORKDIR /root 0.2s
=> [ 3/17] RUN echo APT::Install-Recommends "0"; > /etc/apt/apt.conf.d/10disableextras 0.2s
=> [ 4/17] RUN echo APT::Install-Suggests "0"; >> /etc/apt/apt.conf.d/10disableextras 0.3s
=> [ 5/17] COPY pkg-resolver pkg-resolver 0.0s
=> [ 6/17] RUN chmod a+x pkg-resolver/*.sh pkg-resolver/*.py && chmod a+r pkg-resolver/*.json 0.4s
=> [ 7/17] RUN apt-get -q update && apt-get -q install -y --no-install-recommends python3 && apt-get -q install -y --no-install-recommends $(pkg-resolver/resolve.py ubuntu:focal) && apt-get 79.1s
=> [ 8/17] RUN locale-gen en_US.UTF-8 1.6s
=> [ 9/17] RUN pkg-resolver/install-common-pkgs.sh 2.9s
=> [10/17] RUN pkg-resolver/install-spotbugs.sh ubuntu:focal 1.1s
=> [11/17] RUN pkg-resolver/install-boost.sh ubuntu:focal 290.1s
=> [12/17] RUN pkg-resolver/install-protobuf.sh ubuntu:focal 172.5s
=> [13/17] RUN pkg-resolver/install-hadolint.sh ubuntu:focal 0.8s
=> [14/17] RUN pkg-resolver/install-intel-isa-l.sh ubuntu:focal 15.4s
=> [15/17] COPY hadoop_env_checks.sh /root/hadoop_env_checks.sh 0.1s
=> [16/17] RUN chmod 755 /root/hadoop_env_checks.sh 0.3s
=> [17/17] RUN echo '${HOME}/hadoop_env_checks.sh' >> /root/.bashrc 0.3s
=> exporting to image 12.2s
=> => exporting layers 12.2s
=> => writing image sha256:f443952733cee9b7648897f01d083f19d199b80f0d41523534f6e914df1f7c31 0.0s
=> => naming to docker.io/library/hadoop-build 0.0s
[+] Building 1.5s (9/9) FINISHED docker:default
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 305B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/hadoop-build:latest 0.0s
=> [1/5] FROM docker.io/library/hadoop-build 0.2s
=> [2/5] RUN rm -f /var/log/faillog /var/log/lastlog 0.2s
=> [3/5] RUN groupadd --non-unique -g 1000 sophie 0.3s
=> [4/5] RUN useradd -g 1000 -u 1000 -k /root -m sophie -d "/home/sophie" 0.4s
=> [5/5] RUN echo "sophie ALL=NOPASSWD: ALL" > "/etc/sudoers.d/hadoop-build-1000" 0.3s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:1b3d8fc2cd1437949004b61615dfa5fa68e8142c1fe641ac4b5b1dc462ad5837 0.0s
=> => naming to docker.io/library/hadoop-build-1000 0.0s
_ _ _ ______
| | | | | | | _ \
| |_| | __ _ __| | ___ ___ _ __ | | | |_____ __
| _ |/ _` |/ _` |/ _ \ / _ \| '_ \ | | | / _ \ \ / /
| | | | (_| | (_| | (_) | (_) | |_) | | |/ / __/\ V /
\_| |_/\__,_|\__,_|\___/ \___/| .__/ |___/ \___| \_(_)
| |
|_|
This is the standard Hadoop Developer build environment.
This has all the right tools installed required to build
Hadoop from source.
Following the official build guidance: https://github.com/apache/hadoop/blob/trunk/BUILDING.txt
Prerequisites
Steps
Prepare build environment
git clone -depth 1 git@github.com:apache/hadoop.gitcd hadoop./start-build-env.shBuild