race condition while spawning proxies

[shadeofblue]

When multiple proxies are run within an app and they try to bind to a free port, they find the port already taken by another proxy. As proxy is using a app port from descriptor and binds it to free port on requestor machine, it happens to be already used by another proxy.

We need make proxies not clash into each other, as a race condition, and reserve different free ports.

Example app:

meta:
  name: "Gas scanner demo app"
  author: "GolemFactory"
  version: "0.1.0"
payloads:
  mongo:
    runtime: "vm"
    params:
      image_hash: "69dec7e8e106143d317516580d8e8f36ec014f484eb73b79a042a1f8"
  backend:
    runtime: "vm/manifest"
    params:
      manifest: ewogICAgInZlcnNpb24iOiAiMC4xLjAiLAogICAgImNyZWF0ZWRBdCI6ICIyMDIyLTA5LTI5VDAwOjAwOjAwLjAwMDAwMFoiLAogICAgImV4cGlyZXNBdCI6ICIyMTAwLTAxLTAxVDAwOjAwOjAwLjAwMDAwMFoiLAogICAgInBheWxvYWQiOiBbCiAgICAgICAgewogICAgICAgICAgInBsYXRmb3JtIjogewogICAgICAgICAgICAiYXJjaCI6ICJ4ODZfNjQiLAogICAgICAgICAgICAib3MiOiAibGludXgiCiAgICAgICAgICB9LAogICAgICAgICAgInVybHMiOiBbCiAgICAgICAgICAgICJodHRwOi8veWFjbjIuZGV2LmdvbGVtLm5ldHdvcms6ODAwMC9kb2NrZXItZ2FzX3NjYW5uZXJfYmFja2VuZF9pbWFnZS1sYXRlc3QtM2Y2ZmY2Nzc3My5ndm1pIgogICAgICAgICAgXSwKICAgICAgICAgICJoYXNoIjogInNoYTM6N2U2NzFiYmNiZjE1Njg2Mjc4Y2Y3MmY3Njg0N2FjZjQ0MTQ5ODBhOTkzZTQ4MjczODYwZjIwNmIiCiAgICAgICAgfQogICAgICBdLAogICAgICAiY29tcE1hbmlmZXN0IjogewogICAgICAidmVyc2lvbiI6ICIwLjEuMCIsCiAgICAgICJzY3JpcHQiOiB7CiAgICAgICAgImNvbW1hbmRzIjogWwogICAgICAgICAgInJ1biAuKiIKICAgICAgICBdLAogICAgICAgICJtYXRjaCI6ICJyZWdleCIKICAgICAgfSwKICAgICAgIm5ldCI6IHsKICAgICAgICAiaW5ldCI6IHsKICAgICAgICAgICJvdXQiOiB7CiAgICAgICAgICAgICJwcm90b2NvbHMiOiBbCiAgICAgICAgICAgICAgImh0dHAiCiAgICAgICAgICAgIF0sCiAgICAgICAgICAgICJ1cmxzIjogWwogICAgICAgICAgICAgICAgImh0dHA6Ly9ib3IuZ29sZW0ubmV0d29yayIKICAgICAgICAgICAgXQogICAgICAgICAgfQogICAgICAgIH0KICAgICAgfQogICAgfQogIH0=
  frontend:
    runtime: "vm"
    params:
      image_hash: "49dd38a6da1d6de32f6a7c51f909e104c717bbb27afc24f2beb9da81"
nodes:
  mongo:
    payload: "mongo"
    init:
      - run:
          args: ["/bin/bash", "-c", "/usr/local/bin/docker-entrypoint.sh /usr/bin/mongod --bind_ip_all &"]
    network: "default"
    ip:
      - "192.168.0.2"
    tcp_proxy:
      ports:
        - "27017"
  backend:
    payload: "backend"
    init:
      - run:
          args: [
            "/bin/bash",
            "-c",
            "echo -e \"PROVIDER_ADDRESS=http://bor.golem.network/\n
            MONGO_DB_CONNECTION_STRING=mongodb://192.168.0.2:27017\n
            MONGO_DB_NAME=GasUsage\n
            AGGREGATOR_DELAY_START=60\n
            AGGREGATOR_DELAY_SECONDS='60'\n
            COLD_START_BLOCK='-1'\n
            WATCHDOG_START_AGGREGATOR='node dist/gas_scanner_aggregator.js'\n
            WATCHDOG_START_SERVER='node dist/gas_scanner_server.js'\n
            WATCHDOG_START_COMMAND='node dist/gas_scanner_main.js --fillMissingBlocks --clearDatabase'\n
            WATCHDOG_AFTER_KILL_DELAY_MS=5000\n
            WATCHDOG_AFTER_START_DELAY_MS=15000\n
            WATCHDOG_ALLOWED_SECONDS_BEHIND=30\n
            WATCHDOG_CHECK_EVERY_MS=250000\n
            SERVER_LISTEN_PORT=7888\n
            SERVER_CACHE_VALIDITY=3000\n\" > .env"
          ]
      - run:
          args: ["/bin/bash", "-c", "cp .env dist/.env"]
      - run:
          args: ["/bin/bash", "-c", "node dist/gas_scanner_watchdog.js > stdout 2> stderr &"]
      - run:
          args: ["/bin/bash", "-c", "curl -H \"Content-Type: application/json\" -d '{\"jsonrpc\": \"2.0\", \"method\": \"eth_blockNumber\", \"params\": [], \"id\": 0}' http://bor.golem.network/"]
      - run:
          args: ["/bin/bash", "-c", "sleep 60"]
      - run:
          args: ["/bin/bash", "-c", "cat stdout"]
      - run:
          args: ["/bin/bash", "-c", "cat stderr"]
    http_proxy:
      ports:
        - "7888"
    tcp_proxy:
      ports:
        - "7888"
    network: "default"
    ip:
      - "192.168.0.3"
    depends_on:
      - "mongo"
  frontend:
    payload: "frontend"
    init:
      - run:
          args: ["/bin/bash", "-c", "chmod a+x /"]
      - run:
          args: ["/bin/bash", "-c", "/bin/run.sh 192.168.0.3 &"]
    http_proxy:
      ports:
        - "80"
    network: "default"
    ip:
      - "192.168.0.4"
    depends_on:
      - "backend"
networks:
  default:
    ip: "192.168.0.0/24"

traceback:

[2022-12-06T11:37:50.330+0100 INFO yapapi.summary] Golem engine has shut down
Traceback (most recent call last):
  File "/home/blue/.envs/dapp-runner/bin/dapp-runner", line 6, in <module>
    sys.exit(_cli())
  File "/home/blue/.envs/dapp-runner/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/blue/.envs/dapp-runner/lib/python3.9/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/blue/.envs/dapp-runner/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/blue/.envs/dapp-runner/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/blue/.envs/dapp-runner/lib/python3.9/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/blue/PycharmProjects/dapp-runner/dapp_runner/__main__.py", line 125, in start
    start_runner(config_dict, dapp_dict, **kwargs)
  File "/home/blue/PycharmProjects/dapp-runner/dapp_runner/runner/__init__.py", line 145, in start_runner
    loop.run_until_complete(task)
  File "/home/blue/.pyenv/versions/3.9.12/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete
    return future.result()
  File "/home/blue/PycharmProjects/dapp-runner/dapp_runner/runner/__init__.py", line 105, in _run_app
    await r.stop()
  File "/home/blue/PycharmProjects/dapp-runner/dapp_runner/runner/runner.py", line 300, in stop
    await asyncio.gather(*self._tasks)
  File "/home/blue/PycharmProjects/dapp-runner/dapp_runner/runner/runner.py", line 92, in _start_local_http_proxy
    await proxy.run()
  File "/home/blue/.envs/dapp-runner/lib/python3.9/site-packages/yapapi/contrib/service/http_proxy.py", line 275, in run
    await site.start()
  File "/home/blue/.envs/dapp-runner/lib/python3.9/site-packages/aiohttp/web_runner.py", line 121, in start
    self._server = await loop.create_server(
  File "/home/blue/.pyenv/versions/3.9.12/lib/python3.9/asyncio/base_events.py", line 1506, in create_server
    raise OSError(err.errno, 'error while attempting '
OSError: [Errno 98] error while attempting to bind on address ('0.0.0.0', 8081): address already in use