search

nsacyber / WALKOFF

A flexible, easy to use, automation framework allowing users to integrate their capabilities and devices to cut through the repetitive, tedious tasks slowing them down. #nsacyber
https://nsacyber.github.io/WALKOFF/
Other
1.2k stars 222 forks source link

Running Walkoff using a proxy failes to retrieve images #247

Open crackytsi opened 4 years ago

crackytsi commented 4 years ago

Hi, The latest version does not work, because of missing docker image.

Error: Unable to find image 'walkoff_bootloader:latest' locally docker: Error response from daemon: pull access denied for walkoff_bootloader, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.

g0es commented 4 years ago

make sure you run with the "--build" parameter for the first time, it appears that the bootloader images wasn't built. running up with --build should resolve.

crackytsi commented 4 years ago

@g0es Thanks for your hint, but I did this without succcess. Do you have any futher trick? Are you sure the latest version works for you?

g0es commented 4 years ago

@crackytsi, I am using the latest version and it builds the containers properly on OS X and Ubuntu. What os are you running the build script from? Have you tried to down walkoff with the cleanup flag? I would then clean up docker images associated with Walkoff before building again.

./walkoff.sh down --clean Clean up any docker images that are left from your original attempt. ./walkoff.sh up --build

crackytsi commented 4 years ago

I get the following error Unable to find image 'walkoff_bootloader:latest' locally docker: Error response from daemon: pull access denied for walkoff_bootloader, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.

Details:

Warning: apt-key output should not be parsed (stdout is not a terminal)
Executing: /tmp/apt-key-gpghome.yL8KFtmTdA/gpg.1.sh --fetch-keys https://download.docker.com/linux/debian/gpg
gpg: requesting key from 'https://download.docker.com/linux/debian/gpg'
gpg: WARNING: unable to fetch URI https://download.docker.com/linux/debian/gpg: Connection timed out
Get:2 https://download.docker.com/linux/debian buster InRelease [44.4 kB]
Get:1 http://security-cdn.debian.org/debian-security buster/updates InRelease [65.4 kB]
Hit:3 http://cdn-fastly.deb.debian.org/debian buster InRelease
Err:2 https://download.docker.com/linux/debian buster InRelease
  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 7EA0A9C3F273FCD8
Hit:4 http://cdn-fastly.deb.debian.org/debian buster-updates InRelease
Reading package lists...
W: GPG error: https://download.docker.com/linux/debian buster InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 7EA0A9C3F273FCD8
E: The repository 'https://download.docker.com/linux/debian buster InRelease' is not signed.
The command '/bin/sh -c apt-get update  && apt-get install -y --no-install-recommends     apt-transport-https ca-certificates curl gnupg2 software-properties-common  && apt-key adv --fetch-keys https://download.docker.com/linux/debian/gpg  && add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian $(lsb_release -cs) stable"  && apt-get update  && apt-get install -y --no-install-recommends docker-ce-cli  && rm -rf /var/lib/apt/lists/*' returned a non-zero code: 100
Starting WALKOFF Bootloader...
Unable to find image 'walkoff_bootloader:latest' locally
docker: Error response from daemon: pull access denied for walkoff_bootloader, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.
g0es commented 4 years ago

@crackytsi it looks like the host you are running walkoff on is having issues retrieving https://download.docker.com/linux/debian/gpg can you verify that you can pul the gpg key from that host?

crackytsi commented 4 years ago

@g0es You are right, the issue is related to proxy support. I tried to set a proxy Variables http_proxy and https_proxy but this seems not to be used in all level of walkoff. This proxy variable is used and most steps works (as expected), but later it fails.

I tested it without proxy on an interface facing machine and it works there. So I will try to rename this issue.

adpham95 commented 4 years ago

I tried to set a proxy Variables http_proxy and https_proxy but this seems not to be used in all level of walkoff.

Just to be sure, you're referring to the procedure in https://docs.docker.com/network/proxy/, right? These are handled by Docker, Walkoff itself doesn't know what to do with them.

but later it fails.

Where/when does it fail? Can you share the output/errors?

crackytsi commented 4 years ago

1Yes, I did it exactly this way. Most of the docker parts seems to work, see error message below:

Preparing WALKOFF Bootloader...
Sending build context to Docker daemon  19.04MB
Step 1/11 : FROM python:3.7.4-slim-buster as base
 ---> a8c0694fba17
Step 2/11 : FROM base as builder
 ---> a8c0694fba17
Step 3/11 : RUN apt-get update  && apt-get install -y --no-install-recommends     autoconf g++  && rm -rf /var/lib/apt/lists/*
 ---> Running in 82d34ae9d159
Get:1 http://security-cdn.debian.org/debian-security buster/updates InRelease [65.4 kB]
Get:4 http://security-cdn.debian.org/debian-security buster/updates/main amd64 Packages [161 kB]
Get:2 http://cdn-fastly.deb.debian.org/debian buster InRelease [122 kB]
Get:3 http://cdn-fastly.deb.debian.org/debian buster-updates InRelease [49.3 kB]
Get:5 http://cdn-fastly.deb.debian.org/debian buster/main amd64 Packages [7908 kB]
Get:6 http://cdn-fastly.deb.debian.org/debian buster-updates/main amd64 Packages [5792 B]
Fetched 8311 kB in 14s (589 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
The following additional packages will be installed:
  binutils binutils-common binutils-x86-64-linux-gnu cpp cpp-8 g++-8 gcc gcc-8
  libasan5 libatomic1 libbinutils libc-dev-bin libc6-dev libcc1-0 libgcc-8-dev
  libgdbm-compat4 libgomp1 libisl19 libitm1 liblsan0 libmpc3 libmpfr6 libmpx2
  libperl5.28 libquadmath0 libsigsegv2 libstdc++-8-dev libtsan0 libubsan1
  linux-libc-dev m4 perl perl-modules-5.28
Suggested packages:
  autoconf-archive gnu-standards autoconf-doc libtool gettext binutils-doc
  cpp-doc gcc-8-locales g++-multilib g++-8-multilib gcc-8-doc libstdc++6-8-dbg
  gcc-multilib make manpages-dev automake flex bison gdb gcc-doc
  gcc-8-multilib libgcc1-dbg libgomp1-dbg libitm1-dbg libatomic1-dbg
  libasan5-dbg liblsan0-dbg libtsan0-dbg libubsan1-dbg libmpx2-dbg
  libquadmath0-dbg glibc-doc sensible-utils libstdc++-8-doc m4-doc perl-doc
  libterm-readline-gnu-perl | libterm-readline-perl-perl libb-debug-perl
  liblocale-codes-perl
Recommended packages:
  automake | automaken manpages manpages-dev
The following NEW packages will be installed:
  autoconf binutils binutils-common binutils-x86-64-linux-gnu cpp cpp-8 g++
  g++-8 gcc gcc-8 libasan5 libatomic1 libbinutils libc-dev-bin libc6-dev
  libcc1-0 libgcc-8-dev libgdbm-compat4 libgomp1 libisl19 libitm1 liblsan0
  libmpc3 libmpfr6 libmpx2 libperl5.28 libquadmath0 libsigsegv2
  libstdc++-8-dev libtsan0 libubsan1 linux-libc-dev m4 perl perl-modules-5.28
0 upgraded, 35 newly installed, 0 to remove and 7 not upgraded.
Need to get 50.8 MB of archives.
After this operation, 224 MB of additional disk space will be used.
Get:1 http://security-cdn.debian.org/debian-security buster/updates/main amd64 linux-libc-dev amd64 4.19.67-2+deb10u2 [1234 kB]
Get:2 http://cdn-fastly.deb.debian.org/debian buster/main amd64 perl-modules-5.28 all 5.28.1-6 [2873 kB]
Get:3 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libgdbm-compat4 amd64 1.18.1-4 [44.1 kB]
Get:4 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libperl5.28 amd64 5.28.1-6 [3883 kB]
Get:5 http://cdn-fastly.deb.debian.org/debian buster/main amd64 perl amd64 5.28.1-6 [204 kB]
Get:6 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libsigsegv2 amd64 2.12-2 [32.8 kB]
Get:7 http://cdn-fastly.deb.debian.org/debian buster/main amd64 m4 amd64 1.4.18-2 [203 kB]
Get:8 http://cdn-fastly.deb.debian.org/debian buster/main amd64 autoconf all 2.69-11 [341 kB]
Get:9 http://cdn-fastly.deb.debian.org/debian buster/main amd64 binutils-common amd64 2.31.1-16 [2073 kB]
Get:10 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libbinutils amd64 2.31.1-16 [478 kB]
Get:11 http://cdn-fastly.deb.debian.org/debian buster/main amd64 binutils-x86-64-linux-gnu amd64 2.31.1-16 [1823 kB]
Get:12 http://cdn-fastly.deb.debian.org/debian buster/main amd64 binutils amd64 2.31.1-16 [56.8 kB]
Get:13 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libisl19 amd64 0.20-2 [587 kB]
Get:14 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libmpfr6 amd64 4.0.2-1 [775 kB]
Get:15 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libmpc3 amd64 1.1.0-1 [41.3 kB]
Get:16 http://cdn-fastly.deb.debian.org/debian buster/main amd64 cpp-8 amd64 8.3.0-6 [8914 kB]
Get:17 http://cdn-fastly.deb.debian.org/debian buster/main amd64 cpp amd64 4:8.3.0-1 [19.4 kB]
Get:18 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libcc1-0 amd64 8.3.0-6 [46.6 kB]
Get:19 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libgomp1 amd64 8.3.0-6 [75.8 kB]
Get:20 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libitm1 amd64 8.3.0-6 [27.7 kB]
Get:21 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libatomic1 amd64 8.3.0-6 [9032 B]
Get:22 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libasan5 amd64 8.3.0-6 [362 kB]
Get:23 http://cdn-fastly.deb.debian.org/debian buster/main amd64 liblsan0 amd64 8.3.0-6 [131 kB]
Get:24 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libtsan0 amd64 8.3.0-6 [283 kB]
Get:25 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libubsan1 amd64 8.3.0-6 [120 kB]
Get:26 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libmpx2 amd64 8.3.0-6 [11.4 kB]
Get:27 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libquadmath0 amd64 8.3.0-6 [133 kB]
Get:28 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libgcc-8-dev amd64 8.3.0-6 [2298 kB]
Get:29 http://cdn-fastly.deb.debian.org/debian buster/main amd64 gcc-8 amd64 8.3.0-6 [9452 kB]
Get:30 http://cdn-fastly.deb.debian.org/debian buster/main amd64 gcc amd64 4:8.3.0-1 [5196 B]
Get:31 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libc-dev-bin amd64 2.28-10 [275 kB]
Get:32 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libc6-dev amd64 2.28-10 [2691 kB]
Get:33 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libstdc++-8-dev amd64 8.3.0-6 [1532 kB]
Get:34 http://cdn-fastly.deb.debian.org/debian buster/main amd64 g++-8 amd64 8.3.0-6 [9752 kB]
Get:35 http://cdn-fastly.deb.debian.org/debian buster/main amd64 g++ amd64 4:8.3.0-1 [1644 B]
debconf: delaying package configuration, since apt-utils is not installed
Fetched 50.8 MB in 11s (4617 kB/s)
Selecting previously unselected package perl-modules-5.28.
(Reading database ... 6828 files and directories currently installed.)
Preparing to unpack .../00-perl-modules-5.28_5.28.1-6_all.deb ...
Unpacking perl-modules-5.28 (5.28.1-6) ...
Selecting previously unselected package libgdbm-compat4:amd64.
Preparing to unpack .../01-libgdbm-compat4_1.18.1-4_amd64.deb ...
Unpacking libgdbm-compat4:amd64 (1.18.1-4) ...
Selecting previously unselected package libperl5.28:amd64.
Preparing to unpack .../02-libperl5.28_5.28.1-6_amd64.deb ...
Unpacking libperl5.28:amd64 (5.28.1-6) ...
Selecting previously unselected package perl.
Preparing to unpack .../03-perl_5.28.1-6_amd64.deb ...
Unpacking perl (5.28.1-6) ...
Selecting previously unselected package libsigsegv2:amd64.
Preparing to unpack .../04-libsigsegv2_2.12-2_amd64.deb ...
Unpacking libsigsegv2:amd64 (2.12-2) ...
Selecting previously unselected package m4.
Preparing to unpack .../05-m4_1.4.18-2_amd64.deb ...
Unpacking m4 (1.4.18-2) ...
Selecting previously unselected package autoconf.
Preparing to unpack .../06-autoconf_2.69-11_all.deb ...
Unpacking autoconf (2.69-11) ...
Selecting previously unselected package binutils-common:amd64.
Preparing to unpack .../07-binutils-common_2.31.1-16_amd64.deb ...
Unpacking binutils-common:amd64 (2.31.1-16) ...
Selecting previously unselected package libbinutils:amd64.
Preparing to unpack .../08-libbinutils_2.31.1-16_amd64.deb ...
Unpacking libbinutils:amd64 (2.31.1-16) ...
Selecting previously unselected package binutils-x86-64-linux-gnu.
Preparing to unpack .../09-binutils-x86-64-linux-gnu_2.31.1-16_amd64.deb ...
Unpacking binutils-x86-64-linux-gnu (2.31.1-16) ...
Selecting previously unselected package binutils.
Preparing to unpack .../10-binutils_2.31.1-16_amd64.deb ...
Unpacking binutils (2.31.1-16) ...
Selecting previously unselected package libisl19:amd64.
Preparing to unpack .../11-libisl19_0.20-2_amd64.deb ...
Unpacking libisl19:amd64 (0.20-2) ...
Selecting previously unselected package libmpfr6:amd64.
Preparing to unpack .../12-libmpfr6_4.0.2-1_amd64.deb ...
Unpacking libmpfr6:amd64 (4.0.2-1) ...
Selecting previously unselected package libmpc3:amd64.
Preparing to unpack .../13-libmpc3_1.1.0-1_amd64.deb ...
Unpacking libmpc3:amd64 (1.1.0-1) ...
Selecting previously unselected package cpp-8.
Preparing to unpack .../14-cpp-8_8.3.0-6_amd64.deb ...
Unpacking cpp-8 (8.3.0-6) ...
Selecting previously unselected package cpp.
Preparing to unpack .../15-cpp_4%3a8.3.0-1_amd64.deb ...
Unpacking cpp (4:8.3.0-1) ...
Selecting previously unselected package libcc1-0:amd64.
Preparing to unpack .../16-libcc1-0_8.3.0-6_amd64.deb ...
Unpacking libcc1-0:amd64 (8.3.0-6) ...
Selecting previously unselected package libgomp1:amd64.
Preparing to unpack .../17-libgomp1_8.3.0-6_amd64.deb ...
Unpacking libgomp1:amd64 (8.3.0-6) ...
Selecting previously unselected package libitm1:amd64.
Preparing to unpack .../18-libitm1_8.3.0-6_amd64.deb ...
Unpacking libitm1:amd64 (8.3.0-6) ...
Selecting previously unselected package libatomic1:amd64.
Preparing to unpack .../19-libatomic1_8.3.0-6_amd64.deb ...
Unpacking libatomic1:amd64 (8.3.0-6) ...
Selecting previously unselected package libasan5:amd64.
Preparing to unpack .../20-libasan5_8.3.0-6_amd64.deb ...
Unpacking libasan5:amd64 (8.3.0-6) ...
Selecting previously unselected package liblsan0:amd64.
Preparing to unpack .../21-liblsan0_8.3.0-6_amd64.deb ...
Unpacking liblsan0:amd64 (8.3.0-6) ...
Selecting previously unselected package libtsan0:amd64.
Preparing to unpack .../22-libtsan0_8.3.0-6_amd64.deb ...
Unpacking libtsan0:amd64 (8.3.0-6) ...
Selecting previously unselected package libubsan1:amd64.
Preparing to unpack .../23-libubsan1_8.3.0-6_amd64.deb ...
Unpacking libubsan1:amd64 (8.3.0-6) ...
Selecting previously unselected package libmpx2:amd64.
Preparing to unpack .../24-libmpx2_8.3.0-6_amd64.deb ...
Unpacking libmpx2:amd64 (8.3.0-6) ...
Selecting previously unselected package libquadmath0:amd64.
Preparing to unpack .../25-libquadmath0_8.3.0-6_amd64.deb ...
Unpacking libquadmath0:amd64 (8.3.0-6) ...
Selecting previously unselected package libgcc-8-dev:amd64.
Preparing to unpack .../26-libgcc-8-dev_8.3.0-6_amd64.deb ...
Unpacking libgcc-8-dev:amd64 (8.3.0-6) ...
Selecting previously unselected package gcc-8.
Preparing to unpack .../27-gcc-8_8.3.0-6_amd64.deb ...
Unpacking gcc-8 (8.3.0-6) ...
Selecting previously unselected package gcc.
Preparing to unpack .../28-gcc_4%3a8.3.0-1_amd64.deb ...
Unpacking gcc (4:8.3.0-1) ...
Selecting previously unselected package libc-dev-bin.
Preparing to unpack .../29-libc-dev-bin_2.28-10_amd64.deb ...
Unpacking libc-dev-bin (2.28-10) ...
Selecting previously unselected package linux-libc-dev:amd64.
Preparing to unpack .../30-linux-libc-dev_4.19.67-2+deb10u2_amd64.deb ...
Unpacking linux-libc-dev:amd64 (4.19.67-2+deb10u2) ...
Selecting previously unselected package libc6-dev:amd64.
Preparing to unpack .../31-libc6-dev_2.28-10_amd64.deb ...
Unpacking libc6-dev:amd64 (2.28-10) ...
Selecting previously unselected package libstdc++-8-dev:amd64.
Preparing to unpack .../32-libstdc++-8-dev_8.3.0-6_amd64.deb ...
Unpacking libstdc++-8-dev:amd64 (8.3.0-6) ...
Selecting previously unselected package g++-8.
Preparing to unpack .../33-g++-8_8.3.0-6_amd64.deb ...
Unpacking g++-8 (8.3.0-6) ...
Selecting previously unselected package g++.
Preparing to unpack .../34-g++_4%3a8.3.0-1_amd64.deb ...
Unpacking g++ (4:8.3.0-1) ...
Setting up perl-modules-5.28 (5.28.1-6) ...
Setting up binutils-common:amd64 (2.31.1-16) ...
Setting up linux-libc-dev:amd64 (4.19.67-2+deb10u2) ...
Setting up libgomp1:amd64 (8.3.0-6) ...
Setting up libasan5:amd64 (8.3.0-6) ...
Setting up libmpfr6:amd64 (4.0.2-1) ...
Setting up libsigsegv2:amd64 (2.12-2) ...
Setting up libquadmath0:amd64 (8.3.0-6) ...
Setting up libmpc3:amd64 (1.1.0-1) ...
Setting up libatomic1:amd64 (8.3.0-6) ...
Setting up libgdbm-compat4:amd64 (1.18.1-4) ...
Setting up libperl5.28:amd64 (5.28.1-6) ...
Setting up libmpx2:amd64 (8.3.0-6) ...
Setting up libubsan1:amd64 (8.3.0-6) ...
Setting up libisl19:amd64 (0.20-2) ...
Setting up libbinutils:amd64 (2.31.1-16) ...
Setting up cpp-8 (8.3.0-6) ...
Setting up libc-dev-bin (2.28-10) ...
Setting up libcc1-0:amd64 (8.3.0-6) ...
Setting up liblsan0:amd64 (8.3.0-6) ...
Setting up libitm1:amd64 (8.3.0-6) ...
Setting up binutils-x86-64-linux-gnu (2.31.1-16) ...
Setting up libtsan0:amd64 (8.3.0-6) ...
Setting up m4 (1.4.18-2) ...
Setting up binutils (2.31.1-16) ...
Setting up perl (5.28.1-6) ...
Setting up libgcc-8-dev:amd64 (8.3.0-6) ...
Setting up autoconf (2.69-11) ...
Setting up cpp (4:8.3.0-1) ...
Setting up libc6-dev:amd64 (2.28-10) ...
Setting up libstdc++-8-dev:amd64 (8.3.0-6) ...
Setting up gcc-8 (8.3.0-6) ...
Setting up gcc (4:8.3.0-1) ...
Setting up g++-8 (8.3.0-6) ...
Setting up g++ (4:8.3.0-1) ...
update-alternatives: using /usr/bin/g++ to provide /usr/bin/c++ (c++) in auto mode
Processing triggers for libc-bin (2.28-10) ...
Removing intermediate container 82d34ae9d159
 ---> f40932be81cd
Step 4/11 : RUN mkdir /install
 ---> Running in 7e318cf66a99
Removing intermediate container 7e318cf66a99
 ---> 4d9dec8a1553
Step 5/11 : WORKDIR /install
 ---> Running in 33a291982631
Removing intermediate container 33a291982631
 ---> 34054fa5014c
Step 6/11 : COPY bootloader/requirements.txt /requirements.txt
 ---> fa3f3297f114
Step 7/11 : RUN pip install --no-warn-script-location --prefix="/install" -r /requirements.txt
 ---> Running in 3f4140b90c29
Collecting docker
  Downloading https://files.pythonhosted.org/packages/cc/ca/699d4754a932787ef353a157ada74efd1ceb6d1fc0bfb7989ae1e7b33111/docker-4.1.0-py2.py3-none-any.whl (139kB)
Collecting aiodocker==0.14.0
  Downloading https://files.pythonhosted.org/packages/e2/e4/b8d916221190070b60bf6d60649777ac4c5ec284689404607e41c38bd9b7/aiodocker-0.14.0-py3-none-any.whl
Collecting pyyaml
  Downloading https://files.pythonhosted.org/packages/8d/c9/e5be955a117a1ac548cdd31e37e8fd7b02ce987f9655f5c7563c656d5dcb/PyYAML-5.2.tar.gz (265kB)
Collecting minio
  Downloading https://files.pythonhosted.org/packages/14/46/60bff78df1b112cc50f95c5ffb2e14aaf9aa279a5219845b55c56f214383/minio-5.0.5-py2.py3-none-any.whl (62kB)
Collecting tenacity
  Downloading https://files.pythonhosted.org/packages/bf/d4/8cab4b5239ddb62d950243abff9e88046bb61737ce3eee8694b3d748560c/tenacity-6.0.0-py2.py3-none-any.whl
Collecting requests!=2.18.0,>=2.14.2
  Downloading https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl (57kB)
Collecting six>=1.4.0
  Downloading https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl
Collecting websocket-client>=0.32.0
  Downloading https://files.pythonhosted.org/packages/29/19/44753eab1fdb50770ac69605527e8859468f3c0fd7dc5a76dd9c4dbd7906/websocket_client-0.56.0-py2.py3-none-any.whl (200kB)
Collecting yarl>=1.0
  Downloading https://files.pythonhosted.org/packages/70/54/7cb8eabe7dea1b441a7487ae3a1adb319f2d2e44a062a669f730a24dc474/yarl-1.4.2-cp37-cp37m-manylinux1_x86_64.whl (256kB)
Collecting aiohttp>=2.3.10
  Downloading https://files.pythonhosted.org/packages/e3/71/6000eacb8923d9fd07aa8784a8fab4f022ae697f3c2456d7dca75c743dd6/aiohttp-3.6.2-cp37-cp37m-manylinux1_x86_64.whl (1.2MB)
Collecting python-dateutil
  Downloading https://files.pythonhosted.org/packages/d4/70/d60450c3dd48ef87586924207ae8907090de0b306af2bce5d134d78615cb/python_dateutil-2.8.1-py2.py3-none-any.whl (227kB)
Collecting urllib3
  Downloading https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl (125kB)
Collecting pytz
  Downloading https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl (509kB)
Collecting certifi
  Downloading https://files.pythonhosted.org/packages/b9/63/df50cac98ea0d5b006c55a399c3bf1db9da7b5a24de7890bc9cfd5dd9e99/certifi-2019.11.28-py2.py3-none-any.whl (156kB)
Collecting chardet<3.1.0,>=3.0.2
  Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB)
Collecting idna<2.9,>=2.5
  Downloading https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl (58kB)
Collecting multidict>=4.0
  Downloading https://files.pythonhosted.org/packages/40/de/cc31c25e272be56094ca45bc00396cd30c9c0b2da9ca7b8d206b5269c87f/multidict-4.7.1-cp37-cp37m-manylinux1_x86_64.whl (115kB)
Collecting attrs>=17.3.0
  Downloading https://files.pythonhosted.org/packages/a2/db/4313ab3be961f7a763066401fb77f7748373b6094076ae2bda2806988af6/attrs-19.3.0-py2.py3-none-any.whl
Collecting async-timeout<4.0,>=3.0
  Downloading https://files.pythonhosted.org/packages/e1/1e/5a4441be21b0726c4464f3f23c8b19628372f606755a9d2e46c187e65ec4/async_timeout-3.0.1-py3-none-any.whl
Building wheels for collected packages: pyyaml
  Building wheel for pyyaml (setup.py): started
  Building wheel for pyyaml (setup.py): finished with status 'done'
  Created wheel for pyyaml: filename=PyYAML-5.2-cp37-cp37m-linux_x86_64.whl size=44209 sha256=83e8eec10da2d32e312b6910b9b4bbb7c0194720ce9e339ca2a6109f2d003779
  Stored in directory: /root/.cache/pip/wheels/54/b7/c7/2ada654ee54483c9329871665aaf4a6056c3ce36f29cf66e67
Successfully built pyyaml
Installing collected packages: certifi, urllib3, chardet, idna, requests, six, websocket-client, docker, multidict, yarl, attrs, async-timeout, aiohttp, aiodocker, pyyaml, python-dateutil, pytz, minio, tenacity
Successfully installed aiodocker-0.14.0 aiohttp-3.6.2 async-timeout-3.0.1 attrs-19.3.0 certifi-2019.11.28 chardet-3.0.4 docker-4.1.0 idna-2.8 minio-5.0.5 multidict-4.7.1 python-dateutil-2.8.1 pytz-2019.3 pyyaml-5.2 requests-2.22.0 six-1.13.0 tenacity-6.0.0 urllib3-1.25.7 websocket-client-0.56.0 yarl-1.4.2
WARNING: You are using pip version 19.3; however, version 19.3.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Removing intermediate container 3f4140b90c29
 ---> dc0248fdd086
Step 8/11 : FROM base
 ---> a8c0694fba17
Step 9/11 : RUN apt-get update  && apt-get install -y --no-install-recommends     apt-transport-https ca-certificates curl gnupg2 software-properties-common  && apt-key adv --fetch-keys https://download.docker.com/linux/debian/gpg  && add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian $(lsb_release -cs) stable"  && apt-get update  && apt-get install -y --no-install-recommends docker-ce-cli  && rm -rf /var/lib/apt/lists/*
 ---> Running in 9a711f12c0e3
Get:1 http://security-cdn.debian.org/debian-security buster/updates InRelease [65.4 kB]
Get:2 http://cdn-fastly.deb.debian.org/debian buster InRelease [122 kB]
Get:3 http://cdn-fastly.deb.debian.org/debian buster-updates InRelease [49.3 kB]
Get:4 http://security-cdn.debian.org/debian-security buster/updates/main amd64 Packages [161 kB]
Get:5 http://cdn-fastly.deb.debian.org/debian buster/main amd64 Packages [7908 kB]
Get:6 http://cdn-fastly.deb.debian.org/debian buster-updates/main amd64 Packages [5792 B]
Fetched 8311 kB in 5s (1648 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
ca-certificates is already the newest version (20190110).
The following additional packages will be installed:
  dirmngr distro-info-data gir1.2-glib-2.0 gir1.2-packagekitglib-1.0 gnupg
  gnupg-l10n gnupg-utils gpg gpg-agent gpg-wks-client gpg-wks-server gpgconf
  gpgsm iso-codes libapt-inst2.0 libassuan0 libcurl3-gnutls libcurl4
  libdbus-1-3 libgirepository-1.0-1 libglib2.0-0 libgssapi-krb5-2 libk5crypto3
  libkeyutils1 libkrb5-3 libkrb5support0 libksba8 libldap-2.4-2 libldap-common
  libmpdec2 libnghttp2-14 libnpth0 libpackagekit-glib2-18 libpsl5
  libpython3-stdlib libpython3.7-minimal libpython3.7-stdlib librtmp1
  libsasl2-2 libsasl2-modules-db libssh2-1 lsb-base lsb-release mime-support
  pinentry-curses python-apt-common python3 python3-apt python3-dbus
  python3-gi python3-minimal python3-pycurl python3-software-properties
  python3.7 python3.7-minimal
Suggested packages:
  dbus-user-session libpam-systemd pinentry-gnome3 tor parcimonie xloadimage
  scdaemon isoquery krb5-doc krb5-user lsb pinentry-doc python3-doc python3-tk
  python3-venv python3-apt-dbg python-apt-doc python-dbus-doc python3-dbus-dbg
  libcurl4-gnutls-dev python-pycurl-doc python3-pycurl-dbg python3.7-venv
  python3.7-doc binutils binfmt-support
Recommended packages:
  dbus libglib2.0-data shared-mime-info xdg-user-dirs krb5-locales packagekit
  publicsuffix libsasl2-modules bzip2 file xz-utils unattended-upgrades
The following NEW packages will be installed:
  apt-transport-https curl dirmngr distro-info-data gir1.2-glib-2.0
  gir1.2-packagekitglib-1.0 gnupg gnupg-l10n gnupg-utils gnupg2 gpg gpg-agent
  gpg-wks-client gpg-wks-server gpgconf gpgsm iso-codes libapt-inst2.0
  libassuan0 libcurl3-gnutls libcurl4 libdbus-1-3 libgirepository-1.0-1
  libglib2.0-0 libgssapi-krb5-2 libk5crypto3 libkeyutils1 libkrb5-3
  libkrb5support0 libksba8 libldap-2.4-2 libldap-common libmpdec2
  libnghttp2-14 libnpth0 libpackagekit-glib2-18 libpsl5 libpython3-stdlib
  libpython3.7-minimal libpython3.7-stdlib librtmp1 libsasl2-2
  libsasl2-modules-db libssh2-1 lsb-base lsb-release mime-support
  pinentry-curses python-apt-common python3 python3-apt python3-dbus
  python3-gi python3-minimal python3-pycurl python3-software-properties
  python3.7 python3.7-minimal software-properties-common
0 upgraded, 59 newly installed, 0 to remove and 7 not upgraded.
Need to get 20.3 MB of archives.
After this operation, 74.3 MB of additional disk space will be used.
Get:1 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libpython3.7-minimal amd64 3.7.3-2 [588 kB]
Get:2 http://cdn-fastly.deb.debian.org/debian buster/main amd64 python3.7-minimal amd64 3.7.3-2 [1736 kB]
Get:3 http://cdn-fastly.deb.debian.org/debian buster/main amd64 python3-minimal amd64 3.7.3-1 [36.6 kB]
Get:4 http://cdn-fastly.deb.debian.org/debian buster/main amd64 mime-support all 3.62 [37.2 kB]
Get:5 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libmpdec2 amd64 2.4.2-2 [87.2 kB]
Get:6 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libpython3.7-stdlib amd64 3.7.3-2 [1732 kB]
Get:7 http://cdn-fastly.deb.debian.org/debian buster/main amd64 python3.7 amd64 3.7.3-2 [330 kB]
Get:8 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libpython3-stdlib amd64 3.7.3-1 [20.0 kB]
Get:9 http://cdn-fastly.deb.debian.org/debian buster/main amd64 python3 amd64 3.7.3-1 [61.5 kB]
Get:10 http://cdn-fastly.deb.debian.org/debian buster/main amd64 apt-transport-https all 1.8.2 [149 kB]
Get:11 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libkeyutils1 amd64 1.6-6 [15.0 kB]
Get:12 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libkrb5support0 amd64 1.17-3 [65.6 kB]
Get:13 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libk5crypto3 amd64 1.17-3 [121 kB]
Get:14 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libkrb5-3 amd64 1.17-3 [370 kB]
Get:15 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libgssapi-krb5-2 amd64 1.17-3 [158 kB]
Get:16 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libsasl2-modules-db amd64 2.1.27+dfsg-1 [69.0 kB]
Get:17 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libsasl2-2 amd64 2.1.27+dfsg-1 [106 kB]
Get:18 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libldap-common all 2.4.47+dfsg-3+deb10u1 [89.6 kB]
Get:19 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libldap-2.4-2 amd64 2.4.47+dfsg-3+deb10u1 [225 kB]
Get:20 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libnghttp2-14 amd64 1.36.0-2+deb10u1 [85.0 kB]
Get:21 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libpsl5 amd64 0.20.2-2 [53.7 kB]
Get:22 http://cdn-fastly.deb.debian.org/debian buster/main amd64 librtmp1 amd64 2.4+20151223.gitfa8646d.1-2 [60.5 kB]
Get:23 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libssh2-1 amd64 1.8.0-2.1 [140 kB]
Get:24 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libcurl4 amd64 7.64.0-4 [332 kB]
Get:25 http://cdn-fastly.deb.debian.org/debian buster/main amd64 curl amd64 7.64.0-4 [264 kB]
Get:26 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libassuan0 amd64 2.5.2-1 [49.4 kB]
Get:27 http://cdn-fastly.deb.debian.org/debian buster/main amd64 gpgconf amd64 2.2.12-1+deb10u1 [510 kB]
Get:28 http://cdn-fastly.deb.debian.org/debian buster/main amd64 lsb-base all 10.2019051400 [28.4 kB]
Get:29 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libksba8 amd64 1.3.5-2 [99.7 kB]
Get:30 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libnpth0 amd64 1.6-1 [18.4 kB]
Get:31 http://cdn-fastly.deb.debian.org/debian buster/main amd64 dirmngr amd64 2.2.12-1+deb10u1 [712 kB]
Get:32 http://cdn-fastly.deb.debian.org/debian buster/main amd64 distro-info-data all 0.41+deb10u1 [6500 B]
Get:33 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libglib2.0-0 amd64 2.58.3-2+deb10u2 [1258 kB]
Get:34 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libgirepository-1.0-1 amd64 1.58.3-2 [92.8 kB]
Get:35 http://cdn-fastly.deb.debian.org/debian buster/main amd64 gir1.2-glib-2.0 amd64 1.58.3-2 [143 kB]
Get:36 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libpackagekit-glib2-18 amd64 1.1.12-5 [116 kB]
Get:37 http://cdn-fastly.deb.debian.org/debian buster/main amd64 gir1.2-packagekitglib-1.0 amd64 1.1.12-5 [35.9 kB]
Get:38 http://cdn-fastly.deb.debian.org/debian buster/main amd64 gnupg-l10n all 2.2.12-1+deb10u1 [1010 kB]
Get:39 http://cdn-fastly.deb.debian.org/debian buster/main amd64 gnupg-utils amd64 2.2.12-1+deb10u1 [861 kB]
Get:40 http://cdn-fastly.deb.debian.org/debian buster/main amd64 gpg amd64 2.2.12-1+deb10u1 [865 kB]
Get:41 http://cdn-fastly.deb.debian.org/debian buster/main amd64 pinentry-curses amd64 1.1.0-2 [64.5 kB]
Get:42 http://cdn-fastly.deb.debian.org/debian buster/main amd64 gpg-agent amd64 2.2.12-1+deb10u1 [617 kB]
Get:43 http://cdn-fastly.deb.debian.org/debian buster/main amd64 gpg-wks-client amd64 2.2.12-1+deb10u1 [485 kB]
Get:44 http://cdn-fastly.deb.debian.org/debian buster/main amd64 gpg-wks-server amd64 2.2.12-1+deb10u1 [478 kB]
Get:45 http://cdn-fastly.deb.debian.org/debian buster/main amd64 gpgsm amd64 2.2.12-1+deb10u1 [604 kB]
Get:46 http://cdn-fastly.deb.debian.org/debian buster/main amd64 gnupg all 2.2.12-1+deb10u1 [715 kB]
Get:47 http://cdn-fastly.deb.debian.org/debian buster/main amd64 gnupg2 all 2.2.12-1+deb10u1 [393 kB]
Get:48 http://cdn-fastly.deb.debian.org/debian buster/main amd64 iso-codes all 4.2-1 [2713 kB]
Get:49 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libapt-inst2.0 amd64 1.8.2 [204 kB]
Get:50 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libcurl3-gnutls amd64 7.64.0-4 [329 kB]
Get:51 http://cdn-fastly.deb.debian.org/debian buster/main amd64 libdbus-1-3 amd64 1.12.16-1 [214 kB]
Get:52 http://cdn-fastly.deb.debian.org/debian buster/main amd64 lsb-release all 10.2019051400 [27.5 kB]
Get:53 http://cdn-fastly.deb.debian.org/debian buster/main amd64 python-apt-common all 1.8.4 [94.9 kB]
Get:54 http://cdn-fastly.deb.debian.org/debian buster/main amd64 python3-apt amd64 1.8.4 [182 kB]
Get:55 http://cdn-fastly.deb.debian.org/debian buster/main amd64 python3-dbus amd64 1.2.8-3 [103 kB]
Get:56 http://cdn-fastly.deb.debian.org/debian buster/main amd64 python3-gi amd64 3.30.4-1 [180 kB]
Get:57 http://cdn-fastly.deb.debian.org/debian buster/main amd64 python3-pycurl amd64 7.43.0.2-0.1 [66.3 kB]
Get:58 http://cdn-fastly.deb.debian.org/debian buster/main amd64 python3-software-properties all 0.96.20.2-2 [49.6 kB]
Get:59 http://cdn-fastly.deb.debian.org/debian buster/main amd64 software-properties-common all 0.96.20.2-2 [83.3 kB]
debconf: delaying package configuration, since apt-utils is not installed
Fetched 20.3 MB in 8s (2453 kB/s)
Selecting previously unselected package libpython3.7-minimal:amd64.
(Reading database ... 6828 files and directories currently installed.)
Preparing to unpack .../libpython3.7-minimal_3.7.3-2_amd64.deb ...
Unpacking libpython3.7-minimal:amd64 (3.7.3-2) ...
Selecting previously unselected package python3.7-minimal.
Preparing to unpack .../python3.7-minimal_3.7.3-2_amd64.deb ...
Unpacking python3.7-minimal (3.7.3-2) ...
Setting up libpython3.7-minimal:amd64 (3.7.3-2) ...
Setting up python3.7-minimal (3.7.3-2) ...
Selecting previously unselected package python3-minimal.
(Reading database ... 7069 files and directories currently installed.)
Preparing to unpack .../0-python3-minimal_3.7.3-1_amd64.deb ...
Unpacking python3-minimal (3.7.3-1) ...
Selecting previously unselected package mime-support.
Preparing to unpack .../1-mime-support_3.62_all.deb ...
Unpacking mime-support (3.62) ...
Selecting previously unselected package libmpdec2:amd64.
Preparing to unpack .../2-libmpdec2_2.4.2-2_amd64.deb ...
Unpacking libmpdec2:amd64 (2.4.2-2) ...
Selecting previously unselected package libpython3.7-stdlib:amd64.
Preparing to unpack .../3-libpython3.7-stdlib_3.7.3-2_amd64.deb ...
Unpacking libpython3.7-stdlib:amd64 (3.7.3-2) ...
Selecting previously unselected package python3.7.
Preparing to unpack .../4-python3.7_3.7.3-2_amd64.deb ...
Unpacking python3.7 (3.7.3-2) ...
Selecting previously unselected package libpython3-stdlib:amd64.
Preparing to unpack .../5-libpython3-stdlib_3.7.3-1_amd64.deb ...
Unpacking libpython3-stdlib:amd64 (3.7.3-1) ...
Setting up python3-minimal (3.7.3-1) ...
Selecting previously unselected package python3.
(Reading database ... 7506 files and directories currently installed.)
Preparing to unpack .../00-python3_3.7.3-1_amd64.deb ...
Unpacking python3 (3.7.3-1) ...
Selecting previously unselected package apt-transport-https.
Preparing to unpack .../01-apt-transport-https_1.8.2_all.deb ...
Unpacking apt-transport-https (1.8.2) ...
Selecting previously unselected package libkeyutils1:amd64.
Preparing to unpack .../02-libkeyutils1_1.6-6_amd64.deb ...
Unpacking libkeyutils1:amd64 (1.6-6) ...
Selecting previously unselected package libkrb5support0:amd64.
Preparing to unpack .../03-libkrb5support0_1.17-3_amd64.deb ...
Unpacking libkrb5support0:amd64 (1.17-3) ...
Selecting previously unselected package libk5crypto3:amd64.
Preparing to unpack .../04-libk5crypto3_1.17-3_amd64.deb ...
Unpacking libk5crypto3:amd64 (1.17-3) ...
Selecting previously unselected package libkrb5-3:amd64.
Preparing to unpack .../05-libkrb5-3_1.17-3_amd64.deb ...
Unpacking libkrb5-3:amd64 (1.17-3) ...
Selecting previously unselected package libgssapi-krb5-2:amd64.
Preparing to unpack .../06-libgssapi-krb5-2_1.17-3_amd64.deb ...
Unpacking libgssapi-krb5-2:amd64 (1.17-3) ...
Selecting previously unselected package libsasl2-modules-db:amd64.
Preparing to unpack .../07-libsasl2-modules-db_2.1.27+dfsg-1_amd64.deb ...
Unpacking libsasl2-modules-db:amd64 (2.1.27+dfsg-1) ...
Selecting previously unselected package libsasl2-2:amd64.
Preparing to unpack .../08-libsasl2-2_2.1.27+dfsg-1_amd64.deb ...
Unpacking libsasl2-2:amd64 (2.1.27+dfsg-1) ...
Selecting previously unselected package libldap-common.
Preparing to unpack .../09-libldap-common_2.4.47+dfsg-3+deb10u1_all.deb ...
Unpacking libldap-common (2.4.47+dfsg-3+deb10u1) ...
Selecting previously unselected package libldap-2.4-2:amd64.
Preparing to unpack .../10-libldap-2.4-2_2.4.47+dfsg-3+deb10u1_amd64.deb ...
Unpacking libldap-2.4-2:amd64 (2.4.47+dfsg-3+deb10u1) ...
Selecting previously unselected package libnghttp2-14:amd64.
Preparing to unpack .../11-libnghttp2-14_1.36.0-2+deb10u1_amd64.deb ...
Unpacking libnghttp2-14:amd64 (1.36.0-2+deb10u1) ...
Selecting previously unselected package libpsl5:amd64.
Preparing to unpack .../12-libpsl5_0.20.2-2_amd64.deb ...
Unpacking libpsl5:amd64 (0.20.2-2) ...
Selecting previously unselected package librtmp1:amd64.
Preparing to unpack .../13-librtmp1_2.4+20151223.gitfa8646d.1-2_amd64.deb ...
Unpacking librtmp1:amd64 (2.4+20151223.gitfa8646d.1-2) ...
Selecting previously unselected package libssh2-1:amd64.
Preparing to unpack .../14-libssh2-1_1.8.0-2.1_amd64.deb ...
Unpacking libssh2-1:amd64 (1.8.0-2.1) ...
Selecting previously unselected package libcurl4:amd64.
Preparing to unpack .../15-libcurl4_7.64.0-4_amd64.deb ...
Unpacking libcurl4:amd64 (7.64.0-4) ...
Selecting previously unselected package curl.
Preparing to unpack .../16-curl_7.64.0-4_amd64.deb ...
Unpacking curl (7.64.0-4) ...
Selecting previously unselected package libassuan0:amd64.
Preparing to unpack .../17-libassuan0_2.5.2-1_amd64.deb ...
Unpacking libassuan0:amd64 (2.5.2-1) ...
Selecting previously unselected package gpgconf.
Preparing to unpack .../18-gpgconf_2.2.12-1+deb10u1_amd64.deb ...
Unpacking gpgconf (2.2.12-1+deb10u1) ...
Selecting previously unselected package lsb-base.
Preparing to unpack .../19-lsb-base_10.2019051400_all.deb ...
Unpacking lsb-base (10.2019051400) ...
Selecting previously unselected package libksba8:amd64.
Preparing to unpack .../20-libksba8_1.3.5-2_amd64.deb ...
Unpacking libksba8:amd64 (1.3.5-2) ...
Selecting previously unselected package libnpth0:amd64.
Preparing to unpack .../21-libnpth0_1.6-1_amd64.deb ...
Unpacking libnpth0:amd64 (1.6-1) ...
Selecting previously unselected package dirmngr.
Preparing to unpack .../22-dirmngr_2.2.12-1+deb10u1_amd64.deb ...
Unpacking dirmngr (2.2.12-1+deb10u1) ...
Selecting previously unselected package distro-info-data.
Preparing to unpack .../23-distro-info-data_0.41+deb10u1_all.deb ...
Unpacking distro-info-data (0.41+deb10u1) ...
Selecting previously unselected package libglib2.0-0:amd64.
Preparing to unpack .../24-libglib2.0-0_2.58.3-2+deb10u2_amd64.deb ...
Unpacking libglib2.0-0:amd64 (2.58.3-2+deb10u2) ...
Selecting previously unselected package libgirepository-1.0-1:amd64.
Preparing to unpack .../25-libgirepository-1.0-1_1.58.3-2_amd64.deb ...
Unpacking libgirepository-1.0-1:amd64 (1.58.3-2) ...
Selecting previously unselected package gir1.2-glib-2.0:amd64.
Preparing to unpack .../26-gir1.2-glib-2.0_1.58.3-2_amd64.deb ...
Unpacking gir1.2-glib-2.0:amd64 (1.58.3-2) ...
Selecting previously unselected package libpackagekit-glib2-18:amd64.
Preparing to unpack .../27-libpackagekit-glib2-18_1.1.12-5_amd64.deb ...
Unpacking libpackagekit-glib2-18:amd64 (1.1.12-5) ...
Selecting previously unselected package gir1.2-packagekitglib-1.0.
Preparing to unpack .../28-gir1.2-packagekitglib-1.0_1.1.12-5_amd64.deb ...
Unpacking gir1.2-packagekitglib-1.0 (1.1.12-5) ...
Selecting previously unselected package gnupg-l10n.
Preparing to unpack .../29-gnupg-l10n_2.2.12-1+deb10u1_all.deb ...
Unpacking gnupg-l10n (2.2.12-1+deb10u1) ...
Selecting previously unselected package gnupg-utils.
Preparing to unpack .../30-gnupg-utils_2.2.12-1+deb10u1_amd64.deb ...
Unpacking gnupg-utils (2.2.12-1+deb10u1) ...
Selecting previously unselected package gpg.
Preparing to unpack .../31-gpg_2.2.12-1+deb10u1_amd64.deb ...
Unpacking gpg (2.2.12-1+deb10u1) ...
Selecting previously unselected package pinentry-curses.
Preparing to unpack .../32-pinentry-curses_1.1.0-2_amd64.deb ...
Unpacking pinentry-curses (1.1.0-2) ...
Selecting previously unselected package gpg-agent.
Preparing to unpack .../33-gpg-agent_2.2.12-1+deb10u1_amd64.deb ...
Unpacking gpg-agent (2.2.12-1+deb10u1) ...
Selecting previously unselected package gpg-wks-client.
Preparing to unpack .../34-gpg-wks-client_2.2.12-1+deb10u1_amd64.deb ...
Unpacking gpg-wks-client (2.2.12-1+deb10u1) ...
Selecting previously unselected package gpg-wks-server.
Preparing to unpack .../35-gpg-wks-server_2.2.12-1+deb10u1_amd64.deb ...
Unpacking gpg-wks-server (2.2.12-1+deb10u1) ...
Selecting previously unselected package gpgsm.
Preparing to unpack .../36-gpgsm_2.2.12-1+deb10u1_amd64.deb ...
Unpacking gpgsm (2.2.12-1+deb10u1) ...
Selecting previously unselected package gnupg.
Preparing to unpack .../37-gnupg_2.2.12-1+deb10u1_all.deb ...
Unpacking gnupg (2.2.12-1+deb10u1) ...
Selecting previously unselected package gnupg2.
Preparing to unpack .../38-gnupg2_2.2.12-1+deb10u1_all.deb ...
Unpacking gnupg2 (2.2.12-1+deb10u1) ...
Selecting previously unselected package iso-codes.
Preparing to unpack .../39-iso-codes_4.2-1_all.deb ...
Unpacking iso-codes (4.2-1) ...
Selecting previously unselected package libapt-inst2.0:amd64.
Preparing to unpack .../40-libapt-inst2.0_1.8.2_amd64.deb ...
Unpacking libapt-inst2.0:amd64 (1.8.2) ...
Selecting previously unselected package libcurl3-gnutls:amd64.
Preparing to unpack .../41-libcurl3-gnutls_7.64.0-4_amd64.deb ...
Unpacking libcurl3-gnutls:amd64 (7.64.0-4) ...
Selecting previously unselected package libdbus-1-3:amd64.
Preparing to unpack .../42-libdbus-1-3_1.12.16-1_amd64.deb ...
Unpacking libdbus-1-3:amd64 (1.12.16-1) ...
Selecting previously unselected package lsb-release.
Preparing to unpack .../43-lsb-release_10.2019051400_all.deb ...
Unpacking lsb-release (10.2019051400) ...
Selecting previously unselected package python-apt-common.
Preparing to unpack .../44-python-apt-common_1.8.4_all.deb ...
Unpacking python-apt-common (1.8.4) ...
Selecting previously unselected package python3-apt.
Preparing to unpack .../45-python3-apt_1.8.4_amd64.deb ...
Unpacking python3-apt (1.8.4) ...
Selecting previously unselected package python3-dbus.
Preparing to unpack .../46-python3-dbus_1.2.8-3_amd64.deb ...
Unpacking python3-dbus (1.2.8-3) ...
Selecting previously unselected package python3-gi.
Preparing to unpack .../47-python3-gi_3.30.4-1_amd64.deb ...
Unpacking python3-gi (3.30.4-1) ...
Selecting previously unselected package python3-pycurl.
Preparing to unpack .../48-python3-pycurl_7.43.0.2-0.1_amd64.deb ...
Unpacking python3-pycurl (7.43.0.2-0.1) ...
Selecting previously unselected package python3-software-properties.
Preparing to unpack .../49-python3-software-properties_0.96.20.2-2_all.deb ...
Unpacking python3-software-properties (0.96.20.2-2) ...
Selecting previously unselected package software-properties-common.
Preparing to unpack .../50-software-properties-common_0.96.20.2-2_all.deb ...
Unpacking software-properties-common (0.96.20.2-2) ...
Setting up libksba8:amd64 (1.3.5-2) ...
Setting up lsb-base (10.2019051400) ...
Setting up libkeyutils1:amd64 (1.6-6) ...
Setting up libpsl5:amd64 (0.20.2-2) ...
Setting up mime-support (3.62) ...
Setting up apt-transport-https (1.8.2) ...
Setting up libglib2.0-0:amd64 (2.58.3-2+deb10u2) ...
No schema files found: doing nothing.
Setting up distro-info-data (0.41+deb10u1) ...
Setting up libnghttp2-14:amd64 (1.36.0-2+deb10u1) ...
Setting up libpackagekit-glib2-18:amd64 (1.1.12-5) ...
Setting up libnpth0:amd64 (1.6-1) ...
Setting up libassuan0:amd64 (2.5.2-1) ...
Setting up libapt-inst2.0:amd64 (1.8.2) ...
Setting up libldap-common (2.4.47+dfsg-3+deb10u1) ...
Setting up libkrb5support0:amd64 (1.17-3) ...
Setting up libsasl2-modules-db:amd64 (2.1.27+dfsg-1) ...
Setting up gnupg-l10n (2.2.12-1+deb10u1) ...
Setting up librtmp1:amd64 (2.4+20151223.gitfa8646d.1-2) ...
Setting up libdbus-1-3:amd64 (1.12.16-1) ...
Setting up libk5crypto3:amd64 (1.17-3) ...
Setting up libsasl2-2:amd64 (2.1.27+dfsg-1) ...
Setting up python-apt-common (1.8.4) ...
Setting up gpgconf (2.2.12-1+deb10u1) ...
Setting up libgirepository-1.0-1:amd64 (1.58.3-2) ...
Setting up libssh2-1:amd64 (1.8.0-2.1) ...
Setting up libkrb5-3:amd64 (1.17-3) ...
Setting up libmpdec2:amd64 (2.4.2-2) ...
Setting up iso-codes (4.2-1) ...
Setting up gpg (2.2.12-1+deb10u1) ...
Setting up gnupg-utils (2.2.12-1+deb10u1) ...
Setting up pinentry-curses (1.1.0-2) ...
Setting up libpython3.7-stdlib:amd64 (3.7.3-2) ...
Setting up gpg-agent (2.2.12-1+deb10u1) ...
Setting up gpgsm (2.2.12-1+deb10u1) ...
Setting up libldap-2.4-2:amd64 (2.4.47+dfsg-3+deb10u1) ...
Setting up dirmngr (2.2.12-1+deb10u1) ...
Setting up libgssapi-krb5-2:amd64 (1.17-3) ...
Setting up gir1.2-glib-2.0:amd64 (1.58.3-2) ...
Setting up gpg-wks-server (2.2.12-1+deb10u1) ...
Setting up libcurl4:amd64 (7.64.0-4) ...
Setting up curl (7.64.0-4) ...
Setting up libpython3-stdlib:amd64 (3.7.3-1) ...
Setting up python3.7 (3.7.3-2) ...
Setting up gpg-wks-client (2.2.12-1+deb10u1) ...
Setting up libcurl3-gnutls:amd64 (7.64.0-4) ...
Setting up python3 (3.7.3-1) ...
running python rtupdate hooks for python3.7...
running python post-rtupdate hooks for python3.7...
Setting up gir1.2-packagekitglib-1.0 (1.1.12-5) ...
Setting up python3-pycurl (7.43.0.2-0.1) ...
Setting up python3-gi (3.30.4-1) ...
Setting up gnupg (2.2.12-1+deb10u1) ...
Setting up lsb-release (10.2019051400) ...
Setting up python3-dbus (1.2.8-3) ...
Setting up gnupg2 (2.2.12-1+deb10u1) ...
Setting up python3-apt (1.8.4) ...
Setting up python3-software-properties (0.96.20.2-2) ...
Setting up software-properties-common (0.96.20.2-2) ...
Processing triggers for libc-bin (2.28-10) ...
Warning: apt-key output should not be parsed (stdout is not a terminal)
Executing: /tmp/apt-key-gpghome.VvtUtkz54t/gpg.1.sh --fetch-keys https://download.docker.com/linux/debian/gpg
gpg: requesting key from 'https://download.docker.com/linux/debian/gpg'
gpg: WARNING: unable to fetch URI https://download.docker.com/linux/debian/gpg: Connection timed out
Get:2 https://download.docker.com/linux/debian buster InRelease [44.4 kB]
Get:1 http://security-cdn.debian.org/debian-security buster/updates InRelease [65.4 kB]
Hit:3 http://cdn-fastly.deb.debian.org/debian buster InRelease
Hit:4 http://cdn-fastly.deb.debian.org/debian buster-updates InRelease
Err:2 https://download.docker.com/linux/debian buster InRelease
  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 7EA0A9C3F273FCD8
Reading package lists...
W: GPG error: https://download.docker.com/linux/debian buster InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 7EA0A9C3F273FCD8
E: The repository 'https://download.docker.com/linux/debian buster InRelease' is not signed.
The command '/bin/sh -c apt-get update  && apt-get install -y --no-install-recommends     apt-transport-https ca-certificates curl gnupg2 software-properties-common  && apt-key adv --fetch-keys https://download.docker.com/linux/debian/gpg  && add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian $(lsb_release -cs) stable"  && apt-get update  && apt-get install -y --no-install-recommends docker-ce-cli  && rm -rf /var/lib/apt/lists/*' returned a non-zero code: 100
Starting WALKOFF Bootloader...
Unable to find image 'walkoff_bootloader:latest' locally
docker: Error response from daemon: pull access denied for walkoff_bootloader, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.
See 'docker run --help'.
racco42 commented 4 years ago

If I remember corectly it is a problem of the apt-key, resp. gpg ignoring the proxy settings. I was able to overcome it by modifying the Dockerfile and adding the

--keyserver-options http-proxy= to the apt-key line

https://unix.stackexchange.com/questions/361213/unable-to-add-gpg-key-with-apt-key-behind-a-proxy

fjordhunter commented 4 years ago

I ran into a slew of issues installing WALKOFF behind a corporate proxy. Sometimes it was needing to specify the proxy in the Dockerfile and sometimes it was adding my root ca to the container. Here are a few suggestions that helped me:

ENV http_proxy http://(your proxy):(your proxy port) ENV https_proxy http://(your proxy):(your proxy port) COPY /bootloader/(your root ca).crt /usr/local/share/ca-certificates/(your root ca).crt

In the bootloader Dockerfile, for example, I added those lines in the Copy pip packages and source files section before the RUN commands. Was a bit of a pain, as I had do do this for almost all of the Dockerfiles. I didn't thoroughly document which ones and where I had to do that, lots of trial and error, but hope this suggestion helps.

nusantara-self commented 4 years ago

Hi @crackytsi,

First, thank you all for all of your inputs.

I've ran through the exact same issue and here is how I fixed it.

You need to modify line 23 of /bootloader/Dockerfile and add the following right after apt-key adv --fetch-keys : 

--keyserver-options http-proxy=http://proxy:port

as well as modify the config file of docker located at root/.docker/config.json and add :

,
    "proxies": {
        "default": {
            "httpProxy": "http://proxy:port",
            "httpsProxy": "http://proxy:port",
            "noProxy": "localhost"
        }
    }

The bootloader/Dockerfile will correctly execute once you restart the docker service.

sudo systemctl restart docker

However, this does not work for the rest of the installation. You might encounter same struggles during the UMPIRE step.

So I recommend forcing ENV values in every Dockerfiles as a quick fix. It worked for me.

ENV http_proxy http://proxy:port
ENV https_proxy http://proxy:3128

Here is a list of the files you might need to set ENV values :

 ./api/Dockerfile
./app_sdk/Dockerfile
./apps/adversary_hunting/1.0.0/Dockerfile
./apps/basics/1.0.0/Dockerfile
./apps/hive/1.0.0/Dockerfile
./apps/ip_addr_utils/1.0.0/Dockerfile
./apps/mitre_attack/1.0.0/Dockerfile
./apps/nmap/1.0.0/Dockerfile
./apps/power_shell/1.0.0/Dockerfile
./apps/ssh/1.0.0/Dockerfile
./apps/walk_off/1.0.0/Dockerfile
./bootloader/Dockerfile
./socketio/Dockerfile
./testing/Dockerfile
./umpire/Dockerfile
./worker/Dockerfile