search

sassoftware / sas-container-recipes

A collection of recipes and other resources for building containers that include SAS Viya software.
https://www.sas.com
Apache License 2.0
89 stars 42 forks source link

error while running build.sh #16

Closed sirshg closed 5 years ago

sirshg commented 5 years ago

Describe the bug I am running the build.sh using "viyadep" user after following all the instructions but getting this below error... I have proxy setup and can connect to internet. "docker run hello-world" works fine too.....

Err:47 http://cdn-fastly.deb.debian.org/debian stretch/main amd64 openjdk-8-jre-headless amd64 8u212-b01-1~deb9u1 Hash Sum mismatch Hashes of expected file:

To Reproduce Steps to reproduce the behavior:

  1. Run something with the following arguments and flags 'viyadep@server1$ ./build.sh --type single --zip ./SAS_Viya_deployment_data.zip --addons "auth-demo"'

Expected behavior A clear and concise description of what you expected to happen.

Screenshots If applicable, add screenshots to help explain your problem.

Environment (please complete the applicable information):

Client: Version: 18.09.5 API version: 1.39 Go version: go1.10.8 Git commit: e8ff056 Built: Thu Apr 11 04:43:34 2019 OS/Arch: linux/amd64 Experimental: false

Server: Docker Engine - Community Engine: Version: 18.09.5 API version: 1.39 (minimum version 1.12) Go version: go1.10.8 Git commit: e8ff056 Built: Thu Apr 11 04:13:40 2019 OS/Arch: linux/amd64 Experimental: false

Additional context Anything to do with apt-get permission as I am running the build.sh using non-root user?

g8sman commented 5 years ago

Hi @sirshg, the build script is executing the following when creating the container

docker build . \
    --label sas.recipe=true \
    --label sas.recipe.builder.version=${SAS_DOCKER_TAG} \
    --build-arg USER_UID=${UID} \
    --build-arg DOCKER_GID=${DOCKER_GID} \
    --tag sas-container-recipes-builder:${SAS_DOCKER_TAG} \
    --file Dockerfile \

Here is a suggestion of a way to see if we can narrow down what the issue is. Try running the following:

docker run -it --rm --entrypoint /bin/bash golang:1.12.0
root@17199d742718:/go# apt-get update && apt-get install -y openjdk-8-jdk-headless ansible && rm -rf /var/lib/apt/lists/*

and see if that works. If it does, try the build script again. If it does not, then this would mean that the container is having a tough time reaching out to the package manager to get the packages needed to install in the container.

sirshg commented 5 years ago

Still no luck :(

(UAT) root@devbox# docker run -it --rm --entrypoint /bin/bash golang:1.12.0 root@7dffa3a85134:/go# apt-get update && apt-get install -y openjdk-8-jdk-headless ansible && rm -rf /var/lib/apt/lists/* Err:1 http://security.debian.org/debian-security stretch/updates InRelease Could not resolve 'security.debian.org' Err:2 http://deb.debian.org/debian stretch InRelease Could not resolve 'deb.debian.org' Err:3 http://deb.debian.org/debian stretch-updates InRelease Could not resolve 'deb.debian.org' Reading package lists... Done W: Failed to fetch http://deb.debian.org/debian/dists/stretch/InRelease Could not resolve 'deb.debian.org' W: Failed to fetch http://security.debian.org/debian-security/dists/stretch/updates/InRelease Could not resolve 'security.debian.org' W: Failed to fetch http://deb.debian.org/debian/dists/stretch-updates/InRelease Could not resolve 'deb.debian.org' W: Some index files failed to download. They have been ignored, or old ones used instead. Reading package lists... Done Building dependency tree Reading state information... Done E: Unable to locate package openjdk-8-jdk-headless E: Unable to locate package ansible

ajeffowens commented 5 years ago

This looks like a DNS issue on the surface - what does /etc/resolv.conf look like? Is this mounted in from the host? 

root@0553867c1d8b:/etc# findmnt /etc/resolv.conf
TARGET           SOURCE                                                                                             FSTYPE OPTIONS
/etc/resolv.conf /dev/vdc[/containers/0553867c1d8bca00019d4411c5b6ac03ff9a6f1d363f785bd5da775ea6cc3786/resolv.conf] ext4   rw,relatime,seclabel,data=ordered
g8sman commented 5 years ago

My apologies for the delay in getting back to you. I have not experienced this specific issue with Docker so I asked around. Feedback I got was that it could be something with the Docker networking. You can try stopping the Docker daemon and then restarting it and see if that does the trick. Basically the issue is that the running container is not seeing to the outside world. Are you able to start any basic image install something? If we try another image do you have success?

docker run -it --rm --entrypoint /bin/bash centos:7
yum install -y tree
sirshg commented 5 years ago

I have the http proxy configured in systemd as docker.service, so its all fine for docker to connect to internet. I also have now managed to get the http proxy working inside the container(ref : https://docs.docker.com/network/proxy/#configure-the-docker-client) to connect to the internet inside the container.

So below works fine for me now. Yum install completed successfully.

docker run -it --rm --entrypoint /bin/bash centos:7 yum install -y tree

However, I have now run build.sh again after running below but it failed with the similar error as mentioned before :

docker run -it --rm --entrypoint /bin/bash golang:1.12.0 root@a67ce003c551:/go# apt-get update && apt-get install -y openjdk-8-jdk-headless ansible && rm -rf /var/lib/apt/lists/*

Side question : Why its trying to .deb file when I am working on rhel7?

dlawregiets commented 5 years ago

The golang:1.12.0 image that we use as a base for our build image is a Debian image.

sirshg commented 5 years ago

When I am trying to WGET manually, I see this error, looks like CDN bit playing up? Looks like target site doesn't like us?

(UAT) viyadep@xxxx$ wget --no-check-certificate http://security-cdn.debian.org/debian-security/pool/updates/main/o/openjdk-8/openjdk-8-jre-headless_8u212-b01-1~deb9u1_amd64.deb --2019-05-17 13:10:16-- http://security-cdn.debian.org/debian-security/pool/updates/main/o/openjdk-8/openjdk-8-jre-headless_8u212-b01-1~deb9u1_amd64.deb Resolving xxxx(xxxx)... xx.xx.xx.xxx Connecting to lonbxxxxx.net (lonbxxxx.net)|xx.xx.xx.xxx|:8080... connected. Proxy request sent, awaiting response... 200 OK Length: 27257306 (26M) [application/x-debian-package] Saving to: ‘openjdk-8-jre-headless_8u212-b01-1~deb9u1_amd64.deb’

0% [ ] 28 24.0B/s in 25s

2019-05-17 13:10:50 (1.10 B/s) - Read error at byte 28/27257306 (Connection reset by peer). Retrying.

g8sman commented 5 years ago

I will be honest and say that this does not make much sense to me. For your case we could try switching over to a CentOS based image and see if we get further.

g8sman commented 5 years ago

I am working on a Dockerfile that will use CentOS that I will share once I get it working.

sirshg commented 5 years ago

Thanks very much.

g8sman commented 5 years ago

See if the following works for you.

# Creates an environment for running the SAS Container Recipes tool.
# This Dockerfile is built and run by the `build.sh` script.
FROM centos:7 

ARG USER_UID=1000
ARG DOCKER_GID=997

RUN rpm --rebuilddb; \
    yum install --assumeyes java-1.8.0-openjdk-headless ansible wget git; \
    yum clean all; \
    rm --verbose --recursive --force /root/.cache /var/cache/yum; \
    rm --verbose /etc/security/limits.d/20-nproc.conf; \
    wget https://dl.google.com/go/go1.12.2.linux-amd64.tar.gz && \
    tar -xzf go1.12.2.linux-amd64.tar.gz && \
    mv go /usr/local && \
    groupadd --gid ${DOCKER_GID} docker; \
    useradd --uid ${USER_UID} \
      --gid ${DOCKER_GID} \
      --create-home \
      --shell /bin/bash \
      sas

ENV GOROOT=/usr/local/go
ENV GOPATH=$HOME/work
ENV PATH=$GOROOT/bin:$GOPATH/bin:$PATH

RUN go get -x -u gopkg.in/yaml.v2 github.com/docker/docker/api/types github.com/docker/docker/client

WORKDIR /sas-container-recipes

RUN chmod 777 /sas-container-recipes

COPY --chown=sas:docker addons ./addons
COPY --chown=sas:docker samples ./samples
COPY --chown=sas:docker tests ./tests
COPY --chown=sas:docker util ./util
COPY --chown=sas:docker *.yml *.go ./

RUN chown -R ${USER_GID}:${DOCKER_GID} /sas-container-recipes

USER sas

ENTRYPOINT ["/usr/local/go/bin/go", "run", "main.go", "container.go", "order.go"]
sirshg commented 5 years ago

Thanks for a quick turnaround, I gave it a go. But looks like my server has some issue reaching some of the packages from the internet. It's bit weird as the download works for some and times out for some!

Delta RPMs disabled because /usr/bin/applydeltarpm not installed. warning: /var/cache/yum/x86_64/7/base/packages/copy-jdk-configs-3.3-10.el7_5.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID f4a80eb5: NOKEY Public key for copy-jdk-configs-3.3-10.el7_5.noarch.rpm is not installed Public key for freetype-2.8-12.el7_6.1.x86_64.rpm is not installed Public key for ansible-2.4.2.0-2.el7.noarch.rpm is not installed http://centos.mirroring.pulsant.co.uk/7.6.1810/updates/x86_64/Packages/java-1.8.0-openjdk-headless-1.8.0.212.b04-0.el7_6.x86_64.rpm: [Errno 12] Timeout on http://centos.mirroring.pulsant.co.uk/7.6.1810/updates/x86_64/Packages/java-1.8.0-openjdk-headless-1.8.0.212.b04-0.el7_6.x86_64.rpm: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds') Trying other mirror. http://mirror.as29550.net/mirror.centos.org/7.6.1810/updates/x86_64/Packages/java-1.8.0-openjdk-headless-1.8.0.212.b04-0.el7_6.x86_64.rpm: [Errno 12] Timeout on http://mirror.as29550.net/mirror.centos.org/7.6.1810/updates/x86_64/Packages/java-1.8.0-openjdk-headless-1.8.0.212.b04-0.el7_6.x86_64.rpm: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds') Trying other mirror. http://mirror.bytemark.co.uk/centos/7.6.1810/updates/x86_64/Packages/java-1.8.0-openjdk-headless-1.8.0.212.b04-0.el7_6.x86_64.rpm: [Errno 12] Timeout on http://mirror.bytemark.co.uk/centos/7.6.1810/updates/x86_64/Packages/java-1.8.0-openjdk-headless-1.8.0.212.b04-0.el7_6.x86_64.rpm: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds') Trying other mirror. http://mirror.clustered.net/mirror.centos.org/7.6.1810/updates/x86_64/Packages/java-1.8.0-openjdk-headless-1.8.0.212.b04-0.el7_6.x86_64.rpm: [Errno 12] Timeout on http://mirror.clustered.net/mirror.centos.org/7.6.1810/updates/x86_64/Packages/java-1.8.0-openjdk-headless-1.8.0.212.b04-0.el7_6.x86_64.rpm: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds') Trying other mirror. http://mirror.econdc.com/centos/7.6.1810/updates/x86_64/Packages/java-1.8.0-openjdk-headless-1.8.0.212.b04-0.el7_6.x86_64.rpm: [Errno 12] Timeout on http://mirror.econdc.com/centos/7.6.1810/updates/x86_64/Packages/java-1.8.0-openjdk-headless-1.8.0.212.b04-0.el7_6.x86_64.rpm: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds') Trying other mirror. http://mirror.freethought-internet.co.uk/centos/7.6.1810/updates/x86_64/Packages/java-1.8.0-openjdk-headless-1.8.0.212.b04-0.el7_6.x86_64.rpm: [Errno 12] Timeout on http://mirror.freethought-internet.co.uk/centos/7.6.1810/updates/x86_64/Packages/java-1.8.0-openjdk-headless-1.8.0.212.b04-0.el7_6.x86_64.rpm: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds') Trying other mirror. http://mirror.netweaver.uk/centos/7.6.1810/updates/x86_64/Packages/java-1.8.0-openjdk-headless-1.8.0.212.b04-0.el7_6.x86_64.rpm: [Errno 12] Timeout on http://mirror.netweaver.uk/centos/7.6.1810/updates/x86_64/Packages/java-1.8.0-openjdk-headless-1.8.0.212.b04-0.el7_6.x86_64.rpm: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds') Trying other mirror. http://mirror.sax.uk.as61049.net/centos/7.6.1810/updates/x86_64/Packages/java-1.8.0-openjdk-headless-1.8.0.212.b04-0.el7_6.x86_64.rpm: [Errno 12] Timeout on http://mirror.sax.uk.as61049.net/centos/7.6.1810/updates/x86_64/Packages/java-1.8.0-openjdk-headless-1.8.0.212.b04-0.el7_6.x86_64.rpm: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds') Trying other mirror. http://mirrors.vooservers.com/centos/7.6.1810/updates/x86_64/Packages/java-1.8.0-openjdk-headless-1.8.0.212.b04-0.el7_6.x86_64.rpm: [Errno 12] Timeout on http://mirrors.vooservers.com/centos/7.6.1810/updates/x86_64/Packages/java-1.8.0-openjdk-headless-1.8.0.212.b04-0.el7_6.x86_64.rpm: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds') Trying other mirror. http://www.mirrorservice.org/sites/mirror.centos.org/7.6.1810/updates/x86_64/Packages/java-1.8.0-openjdk-headless-1.8.0.212.b04-0.el7_6.x86_64.rpm: [Errno 12] Timeout on http://www.mirrorservice.org/sites/mirror.centos.org/7.6.1810/updates/x86_64/Packages/java-1.8.0-openjdk-headless-1.8.0.212.b04-0.el7_6.x86_64.rpm: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds') Trying other mirror.

Error downloading packages: 1:java-1.8.0-openjdk-headless-1.8.0.212.b04-0.el7_6.x86_64: [Errno 256] No more mirrors to try.

I will dig more on Monday to find out what's going on if there is anything to do with our network config.

sirshg commented 5 years ago

It has gone further but gets stuck at sas-orchestration level....

2019/05/20 16:07:28 Reading sitedefault.yml ... 2019/05/20 16:07:28 Connecting to the Docker daemon ... 2019/05/20 16:07:28 Fetching orchestration tool ... 2019/05/20 16:07:28 Reading Software Order Email Zip ... 2019/05/20 16:07:28 Skipping loading sitedefault.yml 2019/05/20 16:07:28 Finished connecting to Docker daemon 2019/05/20 16:07:28 Pulling base container image 'centos:7' ... 2019/05/20 16:07:28 Finished reading Software Order Email 2019/05/20 16:07:28 Serving license and entitlement on sas-container-recipes-builder:1976 (172.17.0.2) 2019/05/20 16:07:30 Finished pulling base container image 'centos:7' 2019/05/20 16:08:12 Finished fetching orchestration tool 2019/05/20 16:08:12 Generating playbook for order ... 2019/05/20 16:08:12 Generate playbook output | The build command started 2019/05/20 16:08:15 Generate playbook output | [2019-05-20, 16:08:15.798] IEntitlementsProvider: No entitlements were found. 2019/05/20 16:08:15 Generate playbook output | Error: IEntitlementsProvider: Error loading entitlements from "https://ses.sas.download/ses/entitlements.json". 2019/05/20 16:08:15 Generate playbook output | ses.sas.download 2019/05/20 16:08:15 Generate playbook output |

2019/05/20 16:08:15 [ERROR]: Unable to generate the playbook during cmd.Wait. exit status 1 util/sas-orchestration build --input /SAS_Viya_deployment_data.zip --output builds/single-2019-05-20-16-07-28/sas_viya_playbook.tgz --repository-warehouse https://ses.sas.download/ses/ exit status 1 sas-container-recipes-builder-19.04.0-20190520170715-8522290

I believe this is again a proxy server issue to connect to the internet and sas-orchestration command needs an option to pass the proxy detail to get it working?

I modified the Dockerfile under util to add below option to the sas-orchestration command but no luck!

./sas-orchestration --java-option "-Dhttps.proxyHost=xxxxxxxx" --java-option "-Dhttps.proxyPort=8080" build --input SAS_Viya_deployment_data.zip --deployment-type programming --repository-warehouse $SAS_RPM_REPO_URL --platform $PLATFORM; \

I also used --proxy option for the curl construct command in main.yml file but nothing works.

g8sman commented 5 years ago

I'll have to ask around on this one. You got the same error when making the proxy modifications? Is creating a mirror an option?

sirshg commented 5 years ago

I have mirrored the repository in my local FS and changed Dockerfile under util as :

ARG SAS_RPM_REPO_URL="file:/sastmp/saswork/viya3.4/"

I also changed the order.go with the same detail of the repository but I still see the same error.

2019/05/21 08:16:06 Generate playbook output | The build command started 2019/05/21 08:16:10 Generate playbook output | Error: Error loading entitlements from "file:/sastmp/saswork/viya3.4/entitlements.json". 2019/05/21 08:16:10 Generate playbook output | Error: /sastmp/saswork/viya3.4/entitlements.json (No such file or directory)

2019/05/21 08:16:10 [ERROR]: Unable to generate the playbook during cmd.Wait. exit status 1 util/sas-orchestration build --input /SAS_Viya_deployment_data.zip --output builds/single-2019-05-21-08-15-15/sas_viya_playbook.tgz --repository-warehouse file:/sastmp/saswork/viya3.4/ exit status 1

File does exist but can't find it for some reason....

(UAT) viyadep@xxxxxx$ ls -lrt /sastmp/saswork/viya3.4/entitlements.json -rwxrwxr-x. 1 root root 3420 Sep 12 2018 /sastmp/saswork/viya3.4/entitlements.json

g8sman commented 5 years ago

The file aspect will not work for the container. Is it possible to start up a HTTP server to serve the repository. Let me work on it this morning and see if I can get something that will move us along. I think part of the issue is that we are generating the playbook twice. I think I will create a branch for this issues and then we can play in that branch and see if we can move things along.

sirshg commented 5 years ago

I have configured a HTTP server to serve the local repository and replaced the sas's remote repository link to the one I have. Build.sh has gone further now but still fails with exit status 1... Can't find much detail in the build.log either.

2019/05/21 22:51:30 Connecting to the Docker daemon ... 2019/05/21 22:51:30 Reading Software Order Email Zip ... 2019/05/21 22:51:30 Fetching orchestration tool ... 2019/05/21 22:51:30 Finished connecting to Docker daemon 2019/05/21 22:51:30 Reading sitedefault.yml ... 2019/05/21 22:51:30 Skipping loading sitedefault.yml 2019/05/21 22:51:30 Pulling base container image 'centos:7' ... 2019/05/21 22:51:30 Finished reading Software Order Email 2019/05/21 22:51:30 Serving license and entitlement on sas-container-recipes-builder:1976 (172.17.0.2) 2019/05/21 22:51:32 Finished pulling base container image 'centos:7' 2019/05/21 22:52:25 Finished fetching orchestration tool 2019/05/21 22:52:25 Generating playbook for order ... 2019/05/21 22:52:25 Generate playbook output | The build command started 2019/05/21 22:52:29 Generate playbook output | Examining entitled repositories: 2019/05/21 22:52:33 Generate playbook output | 0%.........10%.........20%.........30%.........40%.........50%.........60%.........70%.........80%.........90%.........100% 2019/05/21 22:52:33 Generate playbook output | Loading deployment definition data: 2019/05/21 22:52:34 Generate playbook output | 0%.........10%.........20%.........30%.........40%.........50%.........60%.........70%.........80%.........90%.........100% 2019/05/21 22:52:34 Generate playbook output | Loading application data: 2019/05/21 22:52:36 Generate playbook output | 0%.........10%.........20%.........30%.........40%.........50%.........60%.........70%.........80%.........90%.........100% 2019/05/21 22:52:43 Generate playbook output | The build command completed successfully 2019/05/21 22:52:43 Extracting generated playbook content ... 2019/05/21 22:52:43 Finished extracting and generating playbook for order 2019/05/21 22:52:43 Fetching the list of containers in the order ... 2019/05/21 22:52:43 Finished fetching the container list 2019/05/21 22:52:43 Skipping adding file to Docker context: addons/auth-demo/Dockerfile 2019/05/21 22:52:43 Skipping adding file to Docker context: addons/auth-demo/addon_config.yml 2019/05/21 22:52:43 Starting 1 build process ... (this may take several minutes) 2019/05/21 22:52:43 [TIP] System resource utilization can be seen by using the docker stats command.

2019/05/21 22:52:43 Starting Docker build: sas-viya-single-programming-only:19.04.0-20190521235115-8522290 ... 2019/05/21 22:52:54 single-programming-only: container build [ERROR] single-programming-only: map[code:1 message:The command '/bin/sh -c set -e; echo; echo "####### Go ahead and add the packages that the Ansible playbook will look for"; echo; if [ "$PLATFORM" = "redhat" ]; then rpm --rebuilddb; yum install --assumeyes java-1.8.0-openjdk libselinux-python which; yum clean all; rm --verbose --recursive --force /root/.cache /var/cache/yum; rm --verbose /etc/security/limits.d/20-nproc.conf; elif [ "$PLATFORM" = "suse" ]; then zypper --non-interactive install -y java-1_8_0-openjdk tar curl hostname iproute2 which gzip; zypper clean --all; rm --verbose --recursive --force /var/cache/zypp; sed -i 's/^/#/g' /etc/security/limits.conf; fi;' returned a non-zero code: 1]

Debugging: builds/single-2019-05-21-22-51-30/sas-viya-single-programming-only/log.txt exit status 1 sas-container-recipes-builder-19.04.0-20190521235115-8522290

g8sman commented 5 years ago

@sirshg, I created a branch issue-16 that has some changes in it to support passing global options along to the build process. Can you pull this branch and see if this helps any? Here is how to pass the options:

 time ./build.sh --type single --zip SAS_Viya_deployment_data.zip --orchestration-global-options https.proxyHost=xxxxxxxx,https.proxyPort=8080

If you can tail builds/single/sas-viya-single-programming-only/log.txt there should be a line like the following

[container.go:395] &{ORCHESTRATION_GLOBAL_OPTIONS=--java-option -Dhttps.proxyHost=xxxxxxxx --java-option -Dhttps.proxyPort=8080  <nil>}

That is there just so we can see if the options came through.

Hope this gets us past this current hurdle.

Cheers!

g8sman commented 5 years ago

In your last run, was there anything in builds/single-2019-05-21-22-51-30/sas-viya-single-programming-only/log.txt?

sirshg commented 5 years ago

I will try the new options....thanks.

log.txt has these messages...

[container.go:391] &{Step 14/32 : RUN set -e; echo; echo "####### Go ahead and add the packages that the Ansible playbook will look for"; echo; if [ "$PLATFORM" = "redhat" ]; then rpm --rebuilddb; yum install --assumeyes java-1.8.0-openjdk libselinux-python which; yum clean all; rm --verbose --recursive --force /root/.cache /var/cache/yum; rm --verbose /etc/security/limits.d/20-nproc.conf; elif [ "$PLATFORM" = "suse" ]; then zypper --non-interactive install -y java-1_8_0-openjdk tar curl hostname iproute2 which gzip; zypper clean --all; rm --verbose --recursive --force /var/cache/zypp; sed -i 's/^/#/g' /etc/security/limits.conf; fi; }

[container.go:391] &{ }

[container.go:391] &{---> Running in 9de28dc8bd69 }

[container.go:391] &{####### Go ahead and add the packages that the Ansible playbook will look for }

[container.go:391] &{Loaded plugins: fastestmirror, ovl }

[container.go:391] &{Determining fastest mirrors }

[container.go:391] &{

One of the configured repositories failed (Unknown), and yum doesn't have enough cached data to continue. At this point the only safe thing yum can do is fail. There are a few ways to work "fix" this:

 1. Contact the upstream for the repository and get them to fix the problem.

 2. Reconfigure the baseurl/etc. for the repository, to point to a working
    upstream. This is most often useful if you are using a newer
    distribution release than is supported by the repository (and the
    packages for the previous distribution release still work).

 3. Run the command with the repository temporarily disabled
        yum --disablerepo=<repoid> ...

 4. Disable the repository permanently, so yum won't use it by default. Yum
    will then just ignore the repository until you permanently enable it
    again or use --enablerepo for temporary usage:

        yum-config-manager --disable <repoid>
    or
        subscription-manager repos --disable=<repoid>

 5. Configure the failing repository to be skipped, if it is unavailable.
    Note that yum will try to contact the repo. when it runs most commands,
    so will have to try and fail each time (and thus. yum will be be much
    slower). If it is a very temporary problem though, this is often a nice
    compromise:

        yum-config-manager --save --setopt=<repoid>.skip_if_unavailable=true

Cannot find a valid baseurl for repo: base/7/x86_64

} [container.go:391] &{Could not retrieve mirrorlist http://mirrorlist.centos.org/?release=7&arch=x86_64&repo=os&infra=container error was 14: curl#6 - "Could not resolve host: mirrorlist.centos.org; Unknown error" } [container.go:391] &{Removing intermediate container 9de28dc8bd69 } [container.go:391] &{Removing intermediate container 9de28dc8bd69 map[code:1 message:The command '/bin/sh -c set -e; echo; echo "####### Go ahead and add the packages that the Ansible playbook will look for"; echo; if [ "$PLATFORM" = "redhat" ]; then rpm --rebuilddb; yum install --assumeyes java-1.8.0-openjdk libselinux-python which; yum clean all; rm --verbose --recursive --force /root/.cache /var/cache/yum; rm --verbose /etc/security/limits.d/20-nproc.conf; elif [ "$PLATFORM" = "suse" ]; then zypper --non-interactive install -y java-1_8_0-openjdk tar curl hostname iproute2 which gzip; zypper clean --all; rm --verbose --recursive --force /var/cache/zypp; sed -i 's/^*/#*/g' /etc/security/limits.conf; fi;' returned a non-zero code: 1]}
g8sman commented 5 years ago

For your particular case, I am thinking the docker-in-docker approach may not work due to the networking issues. If you have a host that you can install docker, golang, java and ansible on, we can skip the docker-in-docker process and see if we can get it working. Whatever networking tricks you did to create the build image, you may need to do it again.

sirshg commented 5 years ago

I have worked out most of the proxy related issues but stuck at java-1.8.0-openjdk-headless install where package download timing out.

http://centos.mirroring.pulsant.co.uk/7.6.1810/updates/x86_64/Packages/java-1.7.0-openjdk-headless-1.7.0.221-2.6.18.0.el7_6.x86_64.rpm: [Errno 12] Timeout on http://centos.mirroring.pulsant.co.uk/7.6.1810/updates/x86_64/Packages/java-1.7.0-openjdk-headless-1.7.0.221-2.6.18.0.el7_6.x86_64.rpm: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds') Trying other mirror. http://centos.mirroring.pulsant.co.uk/7.6.1810/extras/x86_64/Packages/python2-jmespath-0.9.0-3.el7.noarch.rpm: [Errno 12] Timeout on http://centos.mirroring.pulsant.co.uk/7.6.1810/extras/x86_64/Packages/python2-jmespath-0.9.0-3.el7.noarch.rpm: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds') Trying other mirror. http://mirror.as29550.net/mirror.centos.org/7.6.1810/updates/x86_64/Packages/java-1.7.0-openjdk-headless-1.7.0.221-2.6.18.0.el7_6.x86_64.rpm: [Errno 12] Timeout on http://mirror.as29550.net/mirror.centos.org/7.6.1810/updates/x86_64/Packages/java-1.7.0-openjdk-headless-1.7.0.221-2.6.18.0.el7_6.x86_64.rpm: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds') Trying other mirror. http://mirror.bytemark.co.uk/centos/7.6.1810/updates/x86_64/Packages/java-1.7.0-openjdk-headless-1.7.0.221-2.6.18.0.el7_6.x86_64.rpm: [Errno 12] Timeout on http://mirror.bytemark.co.uk/centos/7.6.1810/updates/x86_64/Packages/java-1.7.0-openjdk-headless-1.7.0.221-2.6.18.0.el7_6.x86_64.rpm: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds') Trying other mirror.

If I install java-1.8.0-openjdk-headless on the box itself, how do I make it available in the container instead of yum install in the container? Is that what you meant by saying skipping docker-in-docker process?

g8sman commented 5 years ago

Skipping the docker-in-docker would mean that we would skip https://github.com/sassoftware/sas-container-recipes/blob/master/build.sh#L275 through https://github.com/sassoftware/sas-container-recipes/blob/master/build.sh#L299 and allow the go code to run on the host you are currently on. Hopefully this would simplify some of your setup/configuration.

I would be concerned about trying to mount the java install directory to the underlying image. This would mean that the resulting SAS image would always need that path mounted which is less than ideal.

Another thing you could try is see if you can build your own base image and then use that. This would mean creating a Dockerfile that looks something like

FROM centos:7

ARG PLATFORM=redhat
ENV PLATFORM=$PLATFORM
ARG ANSIBLE_VERSION=2.7
ARG TINI_RPM_NAME=tini_0.18.0.rpm
ARG TINI_URL=https://github.com/krallin/tini/releases/download/v0.18.0

USER root
WORKDIR /tmp/sas

#
# Install some of the expected packages in their own layer
#

RUN set -e; \
    echo; echo "####### Go ahead and add the packages that the Ansible playbook will look for"; echo; \
    if [ "$PLATFORM" = "redhat" ]; then \
        rpm --rebuilddb; \
        yum install --assumeyes java-1.8.0-openjdk libselinux-python which; \
        yum clean all; \
        rm --verbose --recursive --force /root/.cache /var/cache/yum; \
        rm --verbose /etc/security/limits.d/20-nproc.conf; \
    elif [ "$PLATFORM" = "suse" ]; then \
        zypper --non-interactive install -y java-1_8_0-openjdk tar curl hostname iproute2 which gzip; \
        zypper clean --all; \
        rm --verbose --recursive --force /var/cache/zypp; \
        sed -i 's/^*/#*/g' /etc/security/limits.conf; \
    fi;

#
# Install tini to manage the processes
#

RUN set -e; \
    if [ "$PLATFORM" = "redhat" ]; then \
        rpm --rebuilddb; \
        curl --silent --output ${TINI_RPM_NAME} --location ${TINI_URL}/${TINI_RPM_NAME}; \
        yum install --assumeyes ${TINI_RPM_NAME}; \
        rm --verbose ${TINI_RPM_NAME}; \
        yum clean all; \
        rm --verbose --recursive --force /root/.cache /var/cache/yum; \
    elif [ "$PLATFORM" = "suse" ]; then \
        zypper install -y curl ; \
        #Package is unsigned, zypper instead.
        curl --silent --output ${TINI_RPM_NAME} --location ${TINI_URL}/${TINI_RPM_NAME}; \
        rpm -i ${TINI_RPM_NAME}; \
        rm --verbose ${TINI_RPM_NAME}; \
        zypper clean ; \
        rm --verbose --recursive --force /var/cache/zypp; \
    else \
        echo; echo "####### [ERROR] : Unknown platform of \"$PLATFORM\" passed in"; echo; \
        exit 1; \
    fi

#
# Install Ansible and other required packages in their own layer
#

# To get the playbook to run it requires
# + iproute
# + openssh-clients
# + sudo
# + initscripts

RUN set -e; \
    echo; echo "####### Run the Ansible playbook to create the main layer"; echo; \
    if [ "$PLATFORM" = "redhat" ]; then \
        rpm --rebuilddb; \
        echo; echo "####### Add the packages to support running the Ansible playbook"; echo; \
        yum install --assumeyes  https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm ; \
        yum install --assumeyes python-setuptools python-devel openssl-devel; \
        yum install --assumeyes python-pip gcc wget tree automake python-six; \
        pip install --upgrade pip setuptools; \
        pip install ansible==${ANSIBLE_VERSION}; \
        yum install --assumeyes python iproute openssh-clients initscripts sudo; \
    elif [ "$PLATFORM" = "suse" ]; then \
        #Ansible needs \
        zypper --non-interactive install -y python-setuptools python-pyOpenSSL python-devel; \
        #Playbook needs \
        zypper --non-interactive install -y openssh sudo; \
        easy_install pip; \
        zypper remove -y python-cryptography ; \
        pip install ansible==${ANSIBLE_VERSION}; \
    fi;

If you created that file in a directory like /tmp/sas_work_around then the command to build the image would look like

docker build --file Dockerfile . --tag latest

Then you could try building the SAS image passing that image as the base image.

./build.sh --type single --zip /path/to/zip --base-image <name of your image> --base-tag latest --mirror-url <url for mirror>
g8sman commented 5 years ago

I updated the issue-16 branch to have an option to skip the docker in docker approach. If you use this, make sure you have java and golang installed and configured

time ./build.sh --type single --zip SAS_Viya_deployment_data.zip -k --orchestration-global-options https.proxyHost=xxxxxxxx,https.proxyPort=8080 --skip-docker-in-docker

to install golang on my CentOS host I did 

mkdir /tmp/golang
cd /tmp/golang
wget https://dl.google.com/go/go1.12.2.linux-amd64.tar.gz
tar -xzf go1.12.2.linux-amd64.tar.gz
sudo mv go /usr/local
GOROOT=/usr/local/go
GOPATH=$HOME/work
PATH=$GOROOT/bin:$GOPATH/bin:$PATH
go get -x -u gopkg.in/yaml.v2 github.com/docker/docker/api/types github.com/docker/docker/client
sirshg commented 5 years ago

Thanks for all the suggestions, much appreciated. However, I have worked out the master branch itself after modifying Dockerfiles(main image build and util one) to get the proxy thing working and also some of the install including Ansible 2.7 instead of 2.4. I have now managed to work out the java install as well using local repo. This is the snippet of my modified Dockerfile to build the base image :

FROM centos:7

ARG USER_UID=1000 ARG DOCKER_GID=997 ARG ANSIBLE_VERSION=2.7

RUN rpm --rebuilddb; \ yum install --assumeyes http://devxxxxxxx/javarpm/jdk-8u201-linux-x64.rpm; \ yum install --assumeyes https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm; \ yum install --assumeyes python-setuptools python-devel openssl-devel; \ yum install --assumeyes python-pip gcc wget tree automake python-six; \ pip install --upgrade pip setuptools; \ pip install ansible==${ANSIBLE_VERSION}; \ yum install --assumeyes git; \ yum clean all; \

-

Build.sh has now gone much far but errors out with this below msg....

2019/05/22 16:23:46 Generate playbook output | The build command completed successfully 2019/05/22 16:23:46 Extracting generated playbook content ... 2019/05/22 16:23:47 Finished extracting and generating playbook for order 2019/05/22 16:23:47 Fetching the list of containers in the order ... 2019/05/22 16:23:47 Finished fetching the container list 2019/05/22 16:23:47 Skipping adding file to Docker context: addons/auth-demo/Dockerfile 2019/05/22 16:23:47 Skipping adding file to Docker context: addons/auth-demo/addon_config.yml

2019/05/22 16:23:47 Starting 1 build process ... (this may take several minutes) 2019/05/22 16:23:47 [TIP] System resource utilization can be seen by using the docker stats command. 2019/05/22 16:23:47 Starting Docker build: sas-viya-single-programming-only:19.04.0-20190522171751-8522290 ... 2019/05/22 16:34:19 single-programming-only: container build [ERROR] single-programming-only: map[code:2 message:The command '/bin/sh -c set -e; export https_proxy=http://xxxxxx.net:8080; export http_proxy=http://xxxxxx.net:8080; pushd sas_viya_playbook ; echo; echo "####### Run the playbook"; echo; ansible-playbook -i inventory_local.ini site.yml -e '@vars_usermods.yml' -vvv ; echo; echo "####### Stop the running services"; echo; /etc/init.d/sas-viya-all-services stop; popd; echo; echo "####### Reset host variables to localhost"; echo; sed -i 's|^options cashost="." casport|options cashost="localhost" casport|' /opt/sas/viya/config/etc/batchserver/default/autoexec_deployment.sas; sed -i 's|^env.CAS_VIRTUAL_HOST = '.'|env.CAS_VIRTUAL_HOST = 'localhost'|' /opt/sas/viya/config/etc/cas/default/casconfig_deployment.lua; sed -i 's|^SASCONTROLLERHOST=.|SASCONTROLLERHOST=localhost|' /opt/sas/viya/config/etc/sysconfig/cas/default/sas-cas-deployment; sed -i 's|^SAS_CURRENT_HOST=.|SAS_CURRENT_HOST=localhost|' /opt/sas/viya/config/etc/sysconfig/cas/default/sas-cas-deployment; sed -i 's|^options cashost="." casport|options cashost="localhost" casport|' /opt/sas/viya/config/etc/workspaceserver/default/autoexec_deployment.sas; if [ "$PLATFORM" = "redhat" ] ; then sed -i 's|http://.:|http://localhost:|' /etc/httpd/conf.d/proxy.conf; echo "ServerName localhost" >> /etc/httpd/conf/httpd.conf; elif [ "$PLATFORM" = "suse" ]; then sed -i 's|http://.*:|http://localhost:|' /etc/apache2/conf.d/proxy.conf; echo "ServerName localhost" >> /etc/apache2/httpd.conf; fi; echo; echo "####### Remove permstore"; echo; rm --verbose --recursive --force /opt/sas/viya/config/etc/cas/default/permstore/; echo; echo "####### Remove created logs"; echo; rm --verbose --recursive --force /opt/sas/viya/config/var/log/all-services/default/; rm --verbose --recursive --force /opt/sas/viya/config/var/log/cas/default/; rm --verbose --recursive --force /opt/sas/viya/config/var/log/sasstudio/default/; rm --verbose --recursive --force /opt/sas/viya/config/var/log/spawner/default/; echo; if [ "$PLATFORM" = "redhat" ] || [ "$PLATFORM" = "suse" ]; then echo; echo "####### Remove Text Analytic languages; we will add them back in their own layer"; echo; for txtmin in $(rpm -qa | grep sas-txtmin); do if [[ "${txtmin}" != "sas-txtmineng" ]] && [[ "${txtmin}" != "yum" ]]; then echo "####### Uninstalling ${txtmin}"; rpm --verbose -e ${txtmin}; fi; done; fi; if [ "$PLATFORM" = "redhat" ] ; then echo; echo "####### Remove some of the bigger packages that have bloated the layer"; echo; for bigpackage in "sas-mapsgfka1" "sas-mapsgfkb1" "sas-nvidiacuda1" "sas-nvidiacuda" "sas-mapsvahdat" "sas-reportvahdat"; do if rpm -q --quiet ${bigpackage}; then yum erase --assumeyes ${bigpackage}; fi; done; echo; echo "####### Remove the repos"; echo; yum erase --assumeyes "sas-meta-repo"; if [ -e "/etc/yum.repos.d/sas.repo" ]; then rm --verbose /etc/yum.repos.d/sas.repo; fi; echo; echo "####### Clean up yum"; echo; yum clean all; rm --verbose --recursive --force /root/.cache /var/cache/yum; elif [ "$PLATFORM" = "suse" ]; then echo; echo "####### Remove some of the bigger packages that have bloated the layer"; echo; for bigpackage in "sas-mapsgfka1" "sas-mapsgfkb1" "sas-nvidiacuda1" "sas-nvidiacuda" "sas-mapsvahdat" "sas-reportvahdat"; do if rpm -q --quiet ${bigpackage}; then zypper remove -y ${bigpackage}; fi; done; mkdir -p /opt/sas/viya/config/var/log/spawner/default ; chown sas:sas /opt/sas/viya/config/var/log/spawner/default; echo; echo "####### Clean up zypper"; echo; zypper clean --all; rm --verbose --recursive --force /etc/zypp/repos.d/sas-reposuse*.repo; rm --verbose --recursive --force /var/cache/zypp; fi; echo; echo "####### Remove the entitlement certificate"; echo; rm --verbose --recursive --force /etc/pki/sas; echo; echo "####### Remove the content in the WORKDIR"; echo; rm --verbose --recursive --force sas-orchestration sas-orchestration-linux.tgz;' returned a non-zero code: 2]

Debugging: builds/single-2019-05-22-16-22-47/sas-viya-single-programming-only/log.txt exit status 1 sas-container-recipes-builder-19.04.0-20190522171751-8522290

Snippet for log.txt

[container.go:391] &{NO MORE HOSTS LEFT ***** }

[container.go:391] &{PLAY RECAP ***** }

[container.go:391] &{deployTarget : ok=78 changed=14 unreachable=0 failed=1 }

[container.go:391] &{localhost : ok=10 changed=0 unreachable=0 failed=0 }

[container.go:391] &{ }

[container.go:391] &{Removing intermediate container df0dd50bcbd7 }

[container.go:391] &{Removing intermediate container df0dd50bcbd7 map[code:2 message:The command '/bin/sh -c set -e; export https_proxy=http://xxxxxxx.net:8080; export http_proxy=http://xxxxxxx.net:8080; pushd sas_viya_playbook ; echo; echo "####### Run the playbook"; echo; ansible-playbook -i inventory_local.ini site.yml -e '@vars_usermods.yml' -vvv ; echo; echo "####### Stop the running services"; echo; /etc/init.d/sas-viya-all-services stop; popd; echo; echo "####### Reset host variables to localhost"; echo; sed -i 's|^options cashost="." casport|options cashost="localhost" casport|' /opt/sas/viya/config/etc/batchserver/default/autoexec_deployment.sas; sed -i 's|^env.CAS_VIRTUAL_HOST = '.'|env.CAS_VIRTUAL_HOST = 'localhost'|' /opt/sas/viya/config/etc/cas/default/casconfig_deployment.lua; sed -i 's|^SASCONTROLLERHOST=.|SASCONTROLLERHOST=localhost|' /opt/sas/viya/config/etc/sysconfig/cas/default/sas-cas-deployment; sed -i 's|^SAS_CURRENT_HOST=.|SAS_CURRENT_HOST=localhost|' /opt/sas/viya/config/etc/sysconfig/cas/default/sas-cas-deployment; sed -i 's|^options cashost="." casport|options cashost="localhost" casport|' /opt/sas/viya/config/etc/workspaceserver/default/autoexec_deployment.sas; if [ "$PLATFORM" = "redhat" ] ; then sed -i 's|http://.:|http://localhost:|' /etc/httpd/conf.d/proxy.conf; echo "ServerName localhost" >> /etc/httpd/conf/httpd.conf; elif [ "$PLATFORM" = "suse" ]; then sed -i 's|http://.*:|http://localhost:|' /etc/apache2/conf.d/proxy.conf; echo "ServerName localhost" >> /etc/apache2/httpd.conf; fi; echo; echo "####### Remove permstore"; echo; rm --verbose --recursive --force /opt/sas/viya/config/etc/cas/default/permstore/; echo; echo "####### Remove created logs"; echo; rm --verbose --recursive --force /opt/sas/viya/config/var/log/all-services/default/; rm --verbose --recursive --force /opt/sas/viya/config/var/log/cas/default/; rm --verbose --recursive --force /opt/sas/viya/config/var/log/sasstudio/default/; rm --verbose --recursive --force /opt/sas/viya/config/var/log/spawner/default/; echo; if [ "$PLATFORM" = "redhat" ] || [ "$PLATFORM" = "suse" ]; then echo; echo "####### Remove Text Analytic languages; we will add them back in their own layer"; echo; for txtmin in $(rpm -qa | grep sas-txtmin); do if [[ "${txtmin}" != "sas-txtmineng" ]] && [[ "${txtmin}" != "yum" ]]; then echo "####### Uninstalling ${txtmin}"; rpm --verbose -e ${txtmin}; fi; done; fi; if [ "$PLATFORM" = "redhat" ] ; then echo; echo "####### Remove some of the bigger packages that have bloated the layer"; echo; for bigpackage in "sas-mapsgfka1" "sas-mapsgfkb1" "sas-nvidiacuda1" "sas-nvidiacuda" "sas-mapsvahdat" "sas-reportvahdat"; do if rpm -q --quiet ${bigpackage}; then yum erase --assumeyes ${bigpackage}; fi; done; echo; echo "####### Remove the repos"; echo; yum erase --assumeyes "sas-meta-repo"; if [ -e "/etc/yum.repos.d/sas.repo" ]; then rm --verbose /etc/yum.repos.d/sas.repo; fi; echo; echo "####### Clean up yum"; echo; yum clean all; rm --verbose --recursive --force /root/.cache /var/cache/yum; elif [ "$PLATFORM" = "suse" ]; then echo; echo "####### Remove some of the bigger packages that have bloated the layer"; echo; for bigpackage in "sas-mapsgfka1" "sas-mapsgfkb1" "sas-nvidiacuda1" "sas-nvidiacuda" "sas-mapsvahdat" "sas-reportvahdat"; do if rpm -q --quiet ${bigpackage}; then zypper remove -y ${bigpackage}; fi; done; mkdir -p /opt/sas/viya/config/var/log/spawner/default ; chown sas:sas /opt/sas/viya/config/var/log/spawner/default; echo; echo "####### Clean up zypper"; echo; zypper clean --all; rm --verbose --recursive --force /etc/zypp/repos.d/sas-reposuse*.repo; rm --verbose --recursive --force /var/cache/zypp; fi; echo; echo "####### Remove the entitlement certificate"; echo; rm --verbose --recursive --force /etc/pki/sas; echo; echo "####### Remove the content in the WORKDIR"; echo; rm --verbose --recursive --force sas-orchestration sas-orchestration-linux.tgz;' returned a non-zero code: 2]}

Where is it going wrong now?

xujiongda commented 5 years ago

I have the same problem...In my case, ansible says it can't access to the deployTarget(container).

Snippet for log.txt

[container.go:391] &{fatal: [deployTarget]: FAILED! => { "changed": false, "host_keys": {}, "invocation": { "module_args": { "host_list": [ "$containername", "$containerip" ] } }, "msg": "Could not connect to host(s): $containerip, $containername." } }

g8sman commented 5 years ago

@sirshg, can you provide the Ansible output before

[container.go:391] &{NO MORE HOSTS LEFT ************************************************************* }

That will give more information as to what is failing on Ansible.

@xujiongda, I am not sure about the flow and where your system is failing. Can you provide some more detail?

ajeffowens commented 5 years ago

Is anyone here by chance building on a Kubernetes node? I am having similar trouble with containers reaching the outside world and I believe it is because of misconfigured iptables rules. I do not know how to fix it yet, but in my case I can't get past the step where we are downloading sas-orchestration cli. I am able to download it outside of the container, but inside of the container I have no internet connectivity.

sirshg commented 5 years ago

@g8sman - Can you please share your email id so that I can share the log.txt?

xujiongda commented 5 years ago

@g8sman I changed the SAS_Viya_deployment_data.zip and then the issue is solved! Thanks for your help!

sirshg commented 5 years ago

I have finally got it working… phew! I had to export proxy detail for each of the ansible-playbook run section in the util/../Dockerfile itself….

(UAT) viyadep@devxxxxx$ docker images REPOSITORY TAG IMAGE ID CREATED SIZE sas-viya-single-programming-only 19.04.0-20190528105804-8522290 e43a128014f0 About an hour ago 23.8GB sas-container-recipes-builder 19.04.0-20190528105804-8522290 1474f46d3723 2 hours ago 1.57GB centos 7 9f38484d220f 2 months ago 202MB (UAT) viyadep@devxxxxx$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 385d60af626d sas-viya-single-programming-only:19.04.0-20190528105804-8522290 "/usr/bin/tini -- /o…" 17 minutes ago Up 17 minutes 0.0.0.0:5570->5570/tcp, 0.0.0.0:8080->80/tcp, 0.0.0.0:8443->443/tcp sas-programming 2ca643b4a62b sas-viya-single-programming-only:19.04.0-20190528105804-8522290 "/usr/bin/tini -- /o…" 25 minutes ago Up 25 minutes 0.0.0.0:8081->80/tcp, 0.0.0.0:32769->443/tcp, 0.0.0.0:32768->5570/tcp sas-viya-single-programming-only

I have now got the sas studio portal fired up and worked fine with sasdemo user…. Yohoooo!

Thanks for all the help & guidance… much appreciated.

So in summery – these are the changes I made it to work :

1) Used centos7 image instead of golang for base build 2) Used jdk8 using my own local repository to install it in the container 3) Used your code to exclude java install by the ansible-playbook once the jave is there in the container "sed -i 's|^sas_install_java: true|sas_install_java: false|' sas_viya_playbook/vars.yml; \" 4) Used my local mirrored repository to avoid connecting to ses.sas.download/ses for sas orchestration 5) Increased the /var/lib/docker size to 50GB, current usage is ~30GB 6) Exported proxy detail in each of the ansible-playbook run section in the util/…/Dockerfile