search

SenHuang19 / BuildingControlTest

"A Docker setup for building control testing"
3 stars 1 forks source link

Error with docker network name #3

Closed ShantMahserejian closed 2 years ago

ShantMahserejian commented 3 years ago

I keep having an issue when attempting to use docker-compose "up" in parallel. I have included the docker-compose yaml files corresponding to a test that I did attached to this issue (re-saved as .txt files so that GitHub will allow the upload). When I run docker-compose "build" and "up" on the two files in two separate terminal windows, I'mi able to build the docker containers successfully. However, when I attempt to run them in parallel using the Python multiprocessing package, I keep getting an error that a docker network with the same name exists. Below is the output to screen I get when running them in parallel. Please note that I made an attempt to resolve this by having the second run sleep for 3 seconds before running the build and up commands, but the output to screen includes what we see happening for both docker-compose commands at the same time otherwise.

As the attached docker-compose files will show, I even tried to create unique names for each image/container by using my unique simulation names, but the network that keeps being created is called "buildingcontroltest_default" for all cases.

I've also included what a list of the docker images, containers, and networks look like below after the ERROR messages. Please note that I did my best to remove any images, containers, and networks manually, as well as run docker-compose "down" with the options "--remove-orphans" and "--rmi local" to avoid any other issues.

As you can see, the build command works fine. It's the up command that encounters the issue. As a result the docker images and networks are created, but not the containers.

Building eplus_my_teste_1
Step 1/6 : FROM senhuang/eplus8.5
 ---> 24447936ca6c
[docker-compose_my_testE_1.txt](https://github.com/SenHuang19/BuildingControlTest/files/7488395/docker-compose_my_testE_1.txt)
[docker-compose_my_testE_2.txt](https://github.com/SenHuang19/BuildingControlTest/files/7488396/docker-compose_my_testE_2.txt)
Step 2/6 : WORKDIR /home/developer/fmu
 ---> Running in 8393f8167028
Removing intermediate container 8393f8167028
 ---> a0848d179bd0
Step 3/6 : COPY ./eplus /home/developer/fmu/eplus/
 ---> cb754f222488
Step 4/6 : COPY ./web.py /home/developer/fmu/web.py
 ---> 2a4341dcd94f
Step 5/6 : COPY ./testcase.py /home/developer/fmu/testcase.py
 ---> c7cdf97573f5
Step 6/6 : COPY ./config /home/developer/fmu/config
 ---> a8421d43275e
Successfully built a8421d43275e
Successfully tagged buildingcontroltest_eplus_my_teste_1:latest
Building jmodelica_my_teste_1
Step 1/17 : FROM senhuang/jmodelica_flask
 ---> 941e763298ea
Step 2/17 : ENV ROOT_DIR /usr/local
Building eplus_my_teste_2
Step 1/6 : FROM senhuang/eplus8.5
 ---> 24447936ca6c
Step 2/6 : WORKDIR /home/developer/fmu
 ---> Using cache
 ---> a0848d179bd0
Step 3/6 : COPY ./eplus /home/developer/fmu/eplus/
 ---> Using cache
 ---> cb754f222488
Step 4/6 : COPY ./web.py /home/developer/fmu/web.py
 ---> Using cache
 ---> 2a4341dcd94f
Step 5/6 : COPY ./testcase.py /home/developer/fmu/testcase.py
 ---> Using cache
 ---> c7cdf97573f5
Step 6/6 : COPY ./config /home/developer/fmu/config
 ---> Using cache
 ---> a8421d43275e
Successfully built a8421d43275e
Successfully tagged buildingcontroltest_eplus_my_teste_2:latest
Building jmodelica_my_teste_2
Step 1/17 : FROM senhuang/jmodelica_flask
 ---> 941e763298ea
Step 2/17 : ENV ROOT_DIR /usr/local
 ---> Running in 45a5c178c9e3
 ---> Running in 60a79203634a
Removing intermediate container 45a5c178c9e3
 ---> 96e89aa7c782
Step 3/17 : ENV JMODELICA_HOME $ROOT_DIR/JModelica
Removing intermediate container 60a79203634a
 ---> 02a7425d4357
Step 3/17 : ENV JMODELICA_HOME $ROOT_DIR/JModelica
 ---> Running in 74bdf52d2a4a
 ---> Running in 24d3f751ee9b
Removing intermediate container 74bdf52d2a4a
 ---> 68fd2e79cf3b
Step 4/17 : ENV IPOPT_HOME $ROOT_DIR/Ipopt-3.12.4
Removing intermediate container 24d3f751ee9b
 ---> 381794c0c69f
Step 4/17 : ENV IPOPT_HOME $ROOT_DIR/Ipopt-3.12.4
 ---> Running in fd2efca0d422
 ---> Running in ee3420038d63
Removing intermediate container fd2efca0d422
 ---> 6012c368bcce
Step 5/17 : ENV SUNDIALS_HOME $JMODELICA_HOME/ThirdParty/Sundials
Removing intermediate container ee3420038d63
 ---> 4ef61b8d0b7b
Step 5/17 : ENV SUNDIALS_HOME $JMODELICA_HOME/ThirdParty/Sundials
 ---> Running in cb80c1a4829e
 ---> Running in c72beef71c7f
Removing intermediate container cb80c1a4829e
 ---> 6e4465ee919c
Step 6/17 : ENV SEPARATE_PROCESS_JVM /usr/lib/jvm/java-8-openjdk-amd64/
Removing intermediate container c72beef71c7f
 ---> 20d899b5d438
Step 6/17 : ENV SEPARATE_PROCESS_JVM /usr/lib/jvm/java-8-openjdk-amd64/
 ---> Running in 256c67b95e38
 ---> Running in 2675bc815dec
Removing intermediate container 256c67b95e38
 ---> 75142f95a354
Step 7/17 : ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64/
Removing intermediate container 2675bc815dec
 ---> 2152c39d6f55
Step 7/17 : ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64/
 ---> Running in 4e461356748b
 ---> Running in 8ba82c6779b8
Removing intermediate container 4e461356748b
 ---> 74df45d81020
Step 8/17 : ENV PYTHONPATH $PYTHONPATH:$JMODELICA_HOME/Python:$JMODELICA_HOME/Python/pymodelica
Removing intermediate container 8ba82c6779b8
 ---> 73724811b2b5
Step 8/17 : ENV PYTHONPATH $PYTHONPATH:$JMODELICA_HOME/Python:$JMODELICA_HOME/Python/pymodelica
 ---> Running in 55b8200663fd
 ---> Running in 217dea3edefc
Removing intermediate container 55b8200663fd
 ---> 1dfab9fc27ec
Step 9/17 : ENV MODELICAPATH $MODELICAPATH:/home/developer/library
Removing intermediate container 217dea3edefc
 ---> d80da966bb70
Step 9/17 : ENV MODELICAPATH $MODELICAPATH:/home/developer/library
 ---> Running in 96a260ca059b
 ---> Running in 9ad97c9851e4
Removing intermediate container 96a260ca059b
 ---> d42d66dff3fc
Step 10/17 : USER developer
Removing intermediate container 9ad97c9851e4
 ---> e7a37dd142c3
Step 10/17 : USER developer
 ---> Running in bf96a892d65d
 ---> Running in 453eddb6fc82
Removing intermediate container bf96a892d65d
 ---> 5d799b6450b7
Step 11/17 : WORKDIR $HOME
Removing intermediate container 453eddb6fc82
 ---> 7de70710ddf5
Step 11/17 : WORKDIR $HOME
 ---> Running in 4dcc968a90ac
 ---> Running in cddad401927e
Removing intermediate container 4dcc968a90ac
 ---> 886422ff6be4
Step 12/17 : RUN mkdir fmu &&     mkdir library
 ---> Running in 32ba2babd26f
Removing intermediate container cddad401927e
 ---> e772f1b20f4d
Step 12/17 : RUN mkdir fmu &&     mkdir library
 ---> Running in 53b048c44f03
Removing intermediate container 53b048c44f03
 ---> 704f23a0c9f9
Step 13/17 : COPY model/testcase.py $HOME/
Removing intermediate container 32ba2babd26f
 ---> ea925863881d
Step 13/17 : COPY model/testcase.py $HOME/
 ---> 5703076a4653
Step 14/17 : COPY model/web.py $HOME/
 ---> 632b1b2f2198
Step 14/17 : COPY model/web.py $HOME/
 ---> de913ab41f0f
Step 15/17 : COPY model/config $HOME/
 ---> 0b6b21e05616
Step 15/17 : COPY model/config $HOME/
 ---> dffd95f5682b
Step 16/17 : COPY model/fmu $HOME/fmu/
 ---> a1518aa9958f
Step 16/17 : COPY model/fmu $HOME/fmu/
 ---> 6f70160dd01e
Step 17/17 : COPY model/library $HOME/library/
 ---> 90f1182907f9
Step 17/17 : COPY model/library $HOME/library/
 ---> 4970aab64da6
 ---> e0fff05a9d6f
Successfully built e0fff05a9d6f
Successfully built 4970aab64da6
Successfully tagged buildingcontroltest_jmodelica_my_teste_1:latest
Successfully tagged buildingcontroltest_jmodelica_my_teste_2:latest
Creating network "buildingcontroltest_default" with the default driver
Creating network "buildingcontroltest_default" with the default driver
Creating buildingcontroltest_jmodelica_my_teste_2_1 ... 
Creating buildingcontroltest_eplus_my_teste_2_1     ... 
Creating buildingcontroltest_jmodelica_my_teste_1_1 ... 
Creating buildingcontroltest_eplus_my_teste_1_1     ... error

Creating buildingcontroltest_jmodelica_my_teste_2_1 ... error
ound on name)

ERROR: for buildingcontroltest_jmodelica_my_teste_2_1  Cannot start service jmodelica_my_teste_2: network buildingcontroltest_default is ambiguous (2 mCreating buildingcontroltest_eplus_my_teste_2_1     ... error

Creating buildingcontroltest_jmodelica_my_teste_1_1 ... error
ound on name)

ERROR: for jmodelica_my_teste_2  Cannot start service jmodelica_my_teste_2: network buildingcontroltest_default is ambiguous (2 matches found on name)

ERROR: for eplus_my_teste_2  Cannot start service eplus_my_teste_2: network buildingcontroltest_default is ambiguous (2 matches found on name)

ERROR: for buildingcontroltest_jmodelica_my_teste_1_1  Cannot start service jmodelica_my_teste_1: network buildingcontroltest_default is ambiguous (2 matches found on name)
ERROR: Encountered errors while bringing up the project.

ERROR: for eplus_my_teste_1  Cannot start service eplus_my_teste_1: network buildingcontroltest_default is ambiguous (2 matches found on name)

ERROR: for jmodelica_my_teste_1  Cannot start service jmodelica_my_teste_1: network buildingcontroltest_default is ambiguous (2 matches found on name)
ERROR: Encountered errors while bringing up the project.

Traceback (most recent call last):
  File "/home/mahs128/miniconda3/lib/python3.8/site-packages/urllib3/connection.py", line 159, in _new_conn
Traceback (most recent call last):
  File "/home/mahs128/miniconda3/lib/python3.8/site-packages/urllib3/connection.py", line 159, in _new_conn
    conn = connection.create_connection(
    conn = connection.create_connection(
  File "/home/mahs128/miniconda3/lib/python3.8/site-packages/urllib3/util/connection.py", line 84, in create_connection
  File "/home/mahs128/miniconda3/lib/python3.8/site-packages/urllib3/util/connection.py", line 84, in create_connection
    raise err
    raise err
  File "/home/mahs128/miniconda3/lib/python3.8/site-packages/urllib3/util/connection.py", line 74, in create_connection
  File "/home/mahs128/miniconda3/lib/python3.8/site-packages/urllib3/util/connection.py", line 74, in create_connection
    sock.connect(sa)
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
Traceback (most recent call last):
  File "/home/mahs128/miniconda3/lib/python3.8/site-packages/urllib3/connectionpool.py", line 670, in urlopen
  File "/home/mahs128/miniconda3/lib/python3.8/site-packages/urllib3/connectionpool.py", line 670, in urlopen
    httplib_response = self._make_request(
    httplib_response = self._make_request(
  File "/home/mahs128/miniconda3/lib/python3.8/site-packages/urllib3/connectionpool.py", line 392, in _make_request
  File "/home/mahs128/miniconda3/lib/python3.8/site-packages/urllib3/connectionpool.py", line 392, in _make_request
    conn.request(method, url, **httplib_request_kw)
    conn.request(method, url, **httplib_request_kw)
  File "/home/mahs128/miniconda3/lib/python3.8/http/client.py", line 1255, in request
  File "/home/mahs128/miniconda3/lib/python3.8/http/client.py", line 1255, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/mahs128/miniconda3/lib/python3.8/http/client.py", line 1301, in _send_request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/mahs128/miniconda3/lib/python3.8/http/client.py", line 1301, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/mahs128/miniconda3/lib/python3.8/http/client.py", line 1250, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/mahs128/miniconda3/lib/python3.8/http/client.py", line 1010, in _send_output
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/mahs128/miniconda3/lib/python3.8/http/client.py", line 1250, in endheaders
    self.send(msg)
  File "/home/mahs128/miniconda3/lib/python3.8/http/client.py", line 950, in send
    self.connect()
  File "/home/mahs128/miniconda3/lib/python3.8/site-packages/urllib3/connection.py", line 187, in connect
    self._send_output(message_body, encode_chunked=encode_chunked)
    conn = self._new_conn()
  File "/home/mahs128/miniconda3/lib/python3.8/site-packages/urllib3/connection.py", line 171, in _new_conn
  File "/home/mahs128/miniconda3/lib/python3.8/http/client.py", line 1010, in _send_output
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fe0c608e9a0>: Failed to establish a new connection: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mahs128/miniconda3/lib/python3.8/site-packages/requests/adapters.py", line 439, in send
    self.send(msg)
  File "/home/mahs128/miniconda3/lib/python3.8/http/client.py", line 950, in send
    resp = conn.urlopen(
  File "/home/mahs128/miniconda3/lib/python3.8/site-packages/urllib3/connectionpool.py", line 726, in urlopen
    retries = retries.increment(
    self.connect()
  File "/home/mahs128/miniconda3/lib/python3.8/site-packages/urllib3/util/retry.py", line 446, in increment
  File "/home/mahs128/miniconda3/lib/python3.8/site-packages/urllib3/connection.py", line 187, in connect
    conn = self._new_conn()
  File "/home/mahs128/miniconda3/lib/python3.8/site-packages/urllib3/connection.py", line 171, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fcf301159a0>: Failed to establish a new connection: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mahs128/miniconda3/lib/python3.8/site-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/home/mahs128/miniconda3/lib/python3.8/site-packages/urllib3/connectionpool.py", line 726, in urlopen
    retries = retries.increment(
  File "/home/mahs128/miniconda3/lib/python3.8/site-packages/urllib3/util/retry.py", line 446, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='127.0.0.1', port=5003): Max retries exceeded with url: /measurements (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe0c608e9a0>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "run_simulation.py", line 58, in <module>
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='127.0.0.1', port=5001): Max retries exceeded with url: /measurements (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fcf301159a0>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "run_simulation.py", line 58, in <module>
    measurements = requests.get(f'{url_eplus}/measurements').json()
  File "/home/mahs128/miniconda3/lib/python3.8/site-packages/requests/api.py", line 76, in get
    measurements = requests.get(f'{url_eplus}/measurements').json()
  File "/home/mahs128/miniconda3/lib/python3.8/site-packages/requests/api.py", line 76, in get
    return request('get', url, params=params, **kwargs)
    return request('get', url, params=params, **kwargs)
  File "/home/mahs128/miniconda3/lib/python3.8/site-packages/requests/api.py", line 61, in request
  File "/home/mahs128/miniconda3/lib/python3.8/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/mahs128/miniconda3/lib/python3.8/site-packages/requests/sessions.py", line 530, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/mahs128/miniconda3/lib/python3.8/site-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/mahs128/miniconda3/lib/python3.8/site-packages/requests/sessions.py", line 643, in send
    resp = self.send(prep, **send_kwargs)
  File "/home/mahs128/miniconda3/lib/python3.8/site-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/home/mahs128/miniconda3/lib/python3.8/site-packages/requests/adapters.py", line 516, in send
    r = adapter.send(request, **kwargs)
  File "/home/mahs128/miniconda3/lib/python3.8/site-packages/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=5003): Max retries exceeded with url: /measurements (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe0c608e9a0>: Failed to establish a new connection: [Errno 111] Connection refused'))
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=5001): Max retries exceeded with url: /measurements (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fcf301159a0>: Failed to establish a new connection: [Errno 111] Connection refused'))
ERROR: 2 matches found based on name: network buildingcontroltest_default is ambiguous
ERROR: 2 matches found based on name: network buildingcontroltest_default is ambiguous\

(base) mahs128@csledge:~/BuildingControlTest$ docker network ls
NETWORK ID     NAME                          DRIVER    SCOPE
efa838cb9b4e   bridge                        bridge    local
a0106a47dd67   buildingcontroltest_default   bridge    local
5dfe952cab37   buildingcontroltest_default   bridge    local
d3fcec35c118   host                          host      local
1adbd0327f16   mynet                         bridge    local
7342f61eff8e   none                          null      local

(base) mahs128@csledge:~/BuildingControlTest$ docker images
REPOSITORY                                 TAG       IMAGE ID       CREATED          SIZE
buildingcontroltest_jmodelica_my_teste_1   latest    e0fff05a9d6f   39 minutes ago   1.72GB
buildingcontroltest_jmodelica_my_teste_2   latest    4970aab64da6   39 minutes ago   1.72GB
buildingcontroltest_eplus_my_teste_1       latest    a8421d43275e   48 minutes ago   2.1GB
buildingcontroltest_eplus_my_teste_2       latest    a8421d43275e   48 minutes ago   2.1GB
senhuang/eplus8.5                          latest    24447936ca6c   6 months ago     2.1GB
senhuang/jmodelica_flask                   latest    941e763298ea   6 months ago     1.56GB

(base) mahs128@csledge:~/BuildingControlTest$ docker container ls
CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES

docker-compose_my_testE_1.txt docker-compose_my_testE_2.txt

SenHuang19 commented 3 years ago

@ShantMahserejian I could not repeat your error. Did you try to do "docker system prune -a" before running the docker-compose?

ShantMahserejian commented 2 years ago

I resolved the issue by assigning unique network names for the default networks created by docker-compose. I did this by 1) using version: 3.5, and 2) by adding the following to the end of each docker-compose yaml file:

networks:
  default: 
    name: buildingcontroltest_network_{UNIQUE_NAME}

This way, each default network that is created doesn't interfere with other default networks when running in parallel.