CityOfZion / neo-python

Python Node and SDK for the NEO 2.x blockchain. For NEO 3.x go to our successor project neo-mamba
https://neo-python.readthedocs.io/en/latest/
MIT License
313 stars 189 forks source link

Docker build exits with errors #1001

Closed selimerunkut closed 5 years ago

selimerunkut commented 5 years ago

Current behavior

Step 3/10 : RUN apt-get update && apt-get install -y     wget     git-core     python3.6     python3.6-dev     python3.6-venv     python3-pip     libleveldb-dev     libssl-dev     vim     man
 ---> Running in 9a8102f2fa52
Ign:1 http://security.ubuntu.com/ubuntu artful-security InRelease
Ign:2 http://archive.ubuntu.com/ubuntu artful InRelease
Err:3 http://security.ubuntu.com/ubuntu artful-security Release
# Dockerfile to create images running neo-python
  404  Not Found [IP: 91.189.88.31 80]
Ign:4 http://archive.ubuntu.com/ubuntu artful-updates InRelease
Ign:5 http://archive.ubuntu.com/ubuntu artful-backports InRelease
Err:6 http://archive.ubuntu.com/ubuntu artful Release
  404  Not Found [IP: 91.189.88.24 80]
Err:7 http://archive.ubuntu.com/ubuntu artful-updates Release
  404  Not Found [IP: 91.189.88.24 80]
Err:8 http://archive.ubuntu.com/ubuntu artful-backports Release
  404  Not Found [IP: 91.189.88.24 80]
Reading package lists...
E: The repository 'http://security.ubuntu.com/ubuntu artful-security Release' does not have a Release file.
E: The repository 'http://archive.ubuntu.com/ubuntu artful Release' does not have a Release file.
E: The repository 'http://archive.ubuntu.com/ubuntu artful-updates Release' does not have a Release file.
E: The repository 'http://archive.ubuntu.com/ubuntu artful-backports Release' does not have a Release file.
The command '/bin/sh -c apt-get update && apt-get install -y     wget     git-core     python3.6     python3.6-dev     python3.6-venv     python3-pip     libleveldb-dev     libssl-dev     vim     man' returned a non-zero code: 100

Expected behavior

container should be build

How to reproduce

run docker build -t neopython-dev . --build-arg branch=development

Your environment

Mac OSX 10.14.5 Docker Desktop Version 2.0.0.3 (31259)

selimerunkut commented 5 years ago

The solution is to change the release URL and remove the not supported security URLs

Working Docker File:

# Dockerfile to create images running neo-python
# https://github.com/CityOfZion/neo-python/tree/master/docker
#
# Building an image of the current master branch (creates a Docker image called neopython):
#
#    $ docker build -f Dockerfile -t neopython .
#
# Building an image of the current development branch (creates a Docker image called neopython-dev):
#
#    $ docker build -t neopython-dev . --build-arg branch=development
#
# Build without caching:
#
#    $ docker build --no-cache -f Dockerfile -t neopython .
#
# Using with a private network
# -----------------------------
# If you want to run it connecting to a private network, make sure the privatenet container is already running.
# See also https://hub.docker.com/r/cityofzion/neo-privatenet
#
# Start a container interactively, opening a bash in `/neo-python`, and mounting the current directory as `/neo-python/sc`:
#
#    $ docker run --rm -it -v $(pwd):/neo-python/sc --net=host -h neopython --name neopython neopython /bin/bash
#
# Once you are inside the container, you can start neo-python with `np-prompt` (using -p to connect to a private net).
# To update neo-python, just run `git pull` and `pip install -e .`
FROM ubuntu:17.10

# Branch can be overwritten with --build-arg, eg: `--build-arg branch=development`
ARG branch=master

#change source location 
RUN sed -i 's|archive.ubuntu|old-releases.ubuntu|g' /etc/apt/sources.list
#security repos can not be updated and cause errors, remove them
RUN sed -i '/security.ubuntu.com/d' /etc/apt/sources.list

#run a quite update
RUN apt-get -qq update >> /dev/null 2>&1

# Install dependencies
RUN apt-get install -y \
    wget \
    git-core \
    python3.6 \
    python3.6-dev \
    python3.6-venv \
    python3-pip \
    libleveldb-dev \
    libssl-dev \
    vim \
    man

# APT cleanup to reduce image size
RUN rm -rf /var/lib/apt/lists/*

# Clone and setup
RUN git clone https://github.com/CityOfZion/neo-python.git
WORKDIR neo-python
RUN git checkout $branch

# Install the dependencies
RUN pip3 install -e .

# Download the privnet wallet, to have it handy for easy experimenting
RUN wget https://s3.amazonaws.com/neo-experiments/neo-privnet.wallet

# Example run command
CMD /bin/bash
ixje commented 5 years ago

Thanks for the report and solution. I'll give it a go in the next couple days

ixje commented 5 years ago

@nunojusto could you have a quick look and see if this has any impact for your neo-local process? The diff in the docker file is below (left original, right proposed as in the comment above) docker-diff

note that the proposed Docker file works for

docker build -f Dockerfile -t neopython .

but not for

docker build -f Dockerfile -t neopython-dev . --build-arg branch=development

Failing the dev branch might just be because the dev branch now needs python3.7 and thus reports

Step 11/13 : RUN pip3 install -e .
 ---> Running in 19c8e65df054
Obtaining file:///neo-python
neo-python requires Python '>=3.7' but the running Python is 3.6.3
selimerunkut commented 5 years ago

Thanks for the input @ixje and @nunojusto For the branch=development the docker file should be changed to apt-get install python3.7 and so on I would assume.

The proposed Dockerfile is a draft (mainly for my own development purposes) and should be further optimised, maybe also adding a new ubuntu LTS version so we do not need the "sed" hack

ixje commented 5 years ago

Thanks for the tip on fixing the dev branch @overqint I'll wait for a response from @nunojusto before proceeding as Docker ain't my specialty and he's pretty awesome at it as far as I have seen historically.

nunojusto commented 5 years ago

Well, this is related to ubuntu repository problems on the upstream docker image. I've tried to run apt-get update inside a clean ubuntu:17.10 and I receive lots of repositories errors.


$ docker run -it ubuntu:17.10 bash
root@2d904c6273fd:/# 
root@2d904c6273fd:/# 
root@2d904c6273fd:/# apt-get update
Ign:1 http://security.ubuntu.com/ubuntu artful-security InRelease
Ign:2 http://archive.ubuntu.com/ubuntu artful InRelease
Err:3 http://security.ubuntu.com/ubuntu artful-security Release
  404  Not Found [IP: 91.189.88.162 80]
Ign:4 http://archive.ubuntu.com/ubuntu artful-updates InRelease
Ign:5 http://archive.ubuntu.com/ubuntu artful-backports InRelease
Err:6 http://archive.ubuntu.com/ubuntu artful Release
  404  Not Found [IP: 91.189.88.31 80]
Err:7 http://archive.ubuntu.com/ubuntu artful-updates Release
  404  Not Found [IP: 91.189.88.31 80]
Err:8 http://archive.ubuntu.com/ubuntu artful-backports Release
  404  Not Found [IP: 91.189.88.31 80]
Reading package lists... Done
E: The repository 'http://security.ubuntu.com/ubuntu artful-security Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: The repository 'http://archive.ubuntu.com/ubuntu artful Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: The repository 'http://archive.ubuntu.com/ubuntu artful-updates Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: The repository 'http://archive.ubuntu.com/ubuntu artful-backports Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.```

So, don't need to bump the python version for now.
I will file an upstream issue for this, if there isn't one yet.
I'll keep you updated here.
nunojusto commented 5 years ago

UAU... it seems the Ubuntu version we are using is not supported anymore from official Ubuntu Docker hub. Found the reason !!

Supported tags and respective Dockerfile links 18.04, bionic-20190718, bionic, latest 18.10, cosmic-20190719, cosmic 19.04, disco-20190718, disco, rolling 19.10, eoan-20190717.1, eoan, devel 16.04, xenial-20190720, xenial

ixje commented 5 years ago

so we should replace

FROM ubuntu:17.10

with e.g.

FROM ubuntu:18.10

?

nunojusto commented 5 years ago

Someone should approve my MR and then move it to master branch also. And one less issue :)