Unable to reach ganache-cli in a docker container from docker compose

SalahAdDin commented 4 years ago

I asked about this on gitter: [X]
Link to my question on gitter: https://gitter.im/ConsenSys/truffle?at=5f47a4035580fa092b2896e0

Expected Behavior

Both memento and explorer must be connected to ganache-cli.

Current Behavior

For Memento(docker logs):

time="2020-08-27T12:24:40Z" level=info msg="[eth] starting best block tracker"
time="2020-08-27T12:24:40Z" level=info msg="[eth] setting up websocket connection"
time="2020-08-27T12:24:41Z" level=warning msg="error connecting to server: dial tcp 127.0.0.1:8546: connect: connection refused "

For Explorer(in-browser console):

App.ts:25 Couldn't load application config SyntaxError: Unexpected token / in JSON at position 84
    at JSON.parse (<anonymous>)
    at _i.<anonymous> (HttpRequest.ts:101)
    at Generator.next (<anonymous>)
    at a (HttpRequestError.ts:2)

It means they cannot connect properly to ganache-cli.

Steps to Reproduce (for bugs)

Launch up the network: docker-compose -f docker-compose-explorer.yml up
Review the docker-compose logs to see the memento container logs(you can use Kitematic).
Go to the localhost in browser and check the developer console.

Context

This is my docker compose file:

version: "3.7"
services:
  redis:
    container_name: memento-redis
    image: redis:5.0.5-alpine
    restart: always
    command: ["redis-server", "--appendonly", "yes"]
    volumes:
      - .volumes/redis:/data
  postgres:
    container_name: memento-postgres
    image: postgres:11.5-alpine
    restart: always
    environment:
      - POSTGRES_USER=memento
      - POSTGRES_DB=memento
      - POSTGRES_PASSWORD=password
    volumes:
      - .volumes/postgresql:/var/lib/postgresql/data
  ganache-cli:
    image: trufflesuite/ganache-cli
    container_name: ganache-cli
    ports:
      - 8545:8545
      - 8546:8546
  memento:
    container_name: memento
    image: alethio/memento:latest
    restart: always
    depends_on:
      - redis
      - postgres
    ports:
      - 127.0.0.1:3001:3001
      - 127.0.0.1:3000:3000
    environment:
      - PG_PASSWORD=password
    volumes:
      - .volumes/memento:/config
  explorer:
    container_name: memento-lite-explorer
    image: alethio/ethereum-lite-explorer:latest
    restart: always
    depends_on:
      - memento
      - ganache-cli
    ports:
      - 127.0.0.1:80:80
    environment:
      - APP_NODE_URL="http://127.0.0.1:8545"
    volumes:
      - .volumes/lite-explorer/config.json:/usr/share/nginx/html/config.json

Also, i opened an issue in memento repository and i found the next issue.

Your Environment

Version used:v6.10.1 (ganache-core: 2.11.2)
Version of Truffle/Remix/Other tools used:
NodeJS Version: [ ] 6.x, [ ] 7.x (unsupported), [ ] 8.x, [ ] 9.x
Operating System and version (include distro if Linux):

mikeseese commented 4 years ago

it seems that I can't reproduce this as there is missing configuration. What do your various config files look like (i.e. .volumes/memento, .volumes/lite-explorer/config.json, etc)? make sure to sensor out any sensitive information

SalahAdDin commented 4 years ago

I have this in memento configuration file:

# api-related fields
api:
  # The port on which the API will be exposed (default:3001)
  port: 3001

  # Enable/disable development CORS
  dev-cors: true

  # Allowed hosts for HTTP API development CORS
  dev-cors-host: "*"

# Dashboard-related fields
dashboard:
  # The port on which the Dashboard will be exposed (default:3000)
  # WARNING: it must not be the same as api.port!
  port: 3000

  config-management:
    enabled: true

# database fields
db:
  # Database host
  host: "memento-postgres"

  # Database port
  port: 5432

  # Database sslmode
  sslmode: "disable"

  # Database name
  dbname: "memento"

  # Database user (also allowed via PG_USER env var)
  user: "memento"

# ethereum-related fields
eth:
  client:
    # HTTP endpoint of a web3-compatible node (mandatory)
    http: "http://127.0.0.1:8545"

    # Websockets endpoint of JSON-RPC enabled node (optional)
    # if this is present, the task manager will use a newHeads subscription to watch for new blocks
    # if not present, the task manager will use polling to get the highest block
    ws: "ws://127.0.0.1:8546"

    # The interval at which the task manager will poll the node for best block (default:"15s")
    # optional, only used if `ws` url is not specified
    poll-interval: "15s"

# feature flags
feature:
  # Backfilling
  backfill:
    # Enable/disable the backfilling feature
    enabled: false

  # Lag feature
  lag:
    # Enable/disable the lag feature
    enabled: true

    # The amount of blocks to lag behind the tip of the chain if the lag feature is enabled
    value: 10

  # Automatic migrations
  automigrate:
    # Enable/disable the automatic migrations
    enabled: true

  # Uncles scraping
  uncles:
    # Enable/disabled the uncles scraping
    enabled: false

# Control what to be logged using format "module=level,module=level"; `*` means all other modules
logging: "*=info"

# redis-related fields
redis:
  # URL of redis server (default:"localhost:6379")
  server: "memento-redis:6379"

  # The name of the redis list the scraper will use for task management (default: "todo")
  list: "todo"

SalahAdDin commented 4 years ago

And this in the explorer configuration file:

{
  "GOOGLE_ANALYTICS_ID": null,
  "HOTJAR_ID": null,
  "SENTRY_ENABLED": false,

  // "nodeUrl": "http://127.0.0.1:8545",
  // "APP_NODE_URL": "http://localhost:8545",

  "pluginsUrl": "plugins",

  "plugins": [
    {
      "uri": "plugin://aleth.io/eth-common?v=4.0.0"
    },
    {
      "uri": "plugin://aleth.io/3box?v=1.1.1",
      "config": {
        "ipfsUrlMask": "https://ipfs.infura.io/ipfs/%s"
      }
    },
    {
      "uri": "plugin://aleth.io/eth-memento?v=2.0.2",
      "config": {
        "apiBasePath": "http://localhost:3001/api/explorer"
      }
    }
  ],

  "pages": [
    {
      "def": "page://aleth.io/block",
      "children": {
        "sidebar": [
          {
            "def": "context://aleth.io/block/list",
            "children": [{ "def": "module://aleth.io/block/list" }]
          }
        ],
        "content": [
          {
            "def": "module://aleth.io/memento/block/basic",
            "pageCritical": true,
            "children": {
              "confirmations": [
                { "def": "module://aleth.io/block/confirmations" }
              ]
            }
          },
          {
            "def": "module://aleth.io/memento/block/txs",
            "pageCritical": true
          },
          {
            "def": "module://aleth.io/memento/block/advanced",
            "pageCritical": true
          },
          {
            "def": "module://aleth.io/memento/block/logs-bloom",
            "pageCritical": true
          }
        ]
      }
    },
    {
      "def": "page://aleth.io/uncle",
      "children": {
        "content": [
          {
            "def": "module://aleth.io/memento/uncle/details",
            "pageCritical": true
          }
        ]
      }
    },
    {
      "def": "page://aleth.io/tx",
      "children": {
        "sidebar": [
          {
            "def": "context://aleth.io/memento/tx/parentBlock",
            "children": [
              {
                "def": "module://aleth.io/tx/list"
              }
            ]
          }
        ],
        "content": [
          {
            "def": "context://aleth.io/memento/tx/parentBlock",
            "pageCritical": true,
            "children": [
              {
                "def": "module://aleth.io/memento/tx/basic",
                "children": {
                  "blockConfirmations": [
                    {
                      "def": "context://aleth.io/memento/tx/parentBlock",
                      "children": [
                        {
                          "def": "module://aleth.io/block/confirmations"
                        }
                      ]
                    }
                  ]
                }
              },
              {
                "def": "module://aleth.io/memento/tx/advanced"
              },
              {
                "def": "module://aleth.io/memento/tx/summary"
              },
              {
                "def": "module://aleth.io/memento/tx/payload"
              }
            ]
          }
        ]
      }
    },
    {
      "def": "page://aleth.io/account",
      "children": {
        "identicon": [
          {
            "def": "module://aleth.io/3box/profile",
            "children": {
              "fallback": [{ "def": "module://aleth.io/account/identicon" }]
            }
          }
        ],
        "top": [
          {
            "def": "module://aleth.io/memento/account/details",
            "pageCritical": true
          }
        ],
        "bottom": [
          { "def": "module://aleth.io/memento/account/txs" },
          { "def": "module://aleth.io/memento/account/contract" }
        ]
      }
    },
    {
      "def": "page://aleth.io/dashboard",
      "children": {
        "content": [
          { "def": "module://aleth.io/search" },
          {
            "def": "context://aleth.io/dashboard/latestBlockRange",
            "children": [
              {
                "def": "module://aleth.io/dashboard/charts",
                "children": {
                  "content": [
                    {
                      "def": "module://aleth.io/dashboard/blocksChart",
                      "children": {
                        "children": [
                          {
                            "def": "context://aleth.io/block/latest",
                            "children": [
                              {
                                "def": "module://aleth.io/dashboard/latestBlockInfo"
                              }
                            ]
                          }
                        ]
                      }
                    }
                  ]
                }
              }
            ]
          }
        ]
      }
    }
  ],

  "rootModules": {
    "toolbarTop": [
      {
        "def": "module://aleth.io/toolbar/search"
      }
    ],
    "toolbarBottom": [
      {
        "def": "module://aleth.io/toolbar/feedback"
      }
    ],
    "topbar": [
      {
        "def": "module://aleth.io/topbar/search"
      },
      {
        "def": "module://aleth.io/topbar/feedback"
      }
    ]
  }
}

I followed the tutorials, i copied the default's configuration and I changed just what i need to.

davidmurdoch commented 4 years ago

Ganache listens only on a single port at a time (8545 by default). The config here looks like it expects http to be over 8545 only and websockets to be over 8546 only. Try changing the ws: "ws://127.0.0.1:8546" line tows: "ws://127.0.0.1:8545"`.

mikeseese commented 4 years ago

@SalahAdDin In addition to the 8545/8546 port stuff @davidmurdoch mentioned, it looks like the core of your issues circulates around incorrect docker configuration/misunderstanding how docker works rather than issues with ganache or memento

In both the explorer and memento services, you point to ganache using 127.0.0.1. Each of these servers should be considered as separate machines on the same network. Ganache isn't running on the memento machine for example, it's running on the ganache-cli machine/service

Docker lets you refer to them by name, so you just need to change .volumes/memento/config.yml to have http://ganache-cli:8545 for both the http and ws urls. You also need to change your APP_NODE_URL environment variable for the explorer service in your docker compose to - APP_NODE_URL="http://ganache-cli:8545"

Further, the error that you receive in the webapp:

App.ts:25 Couldn't load application config SyntaxError: Unexpected token / in JSON at position 84
   at JSON.parse (<anonymous>)
   at _i.<anonymous> (HttpRequest.ts:101)
   at Generator.next (<anonymous>)
   at a (HttpRequestError.ts:2)

Is happening because your config.json is not valid JSON. JSON doesn't contain comments, so the JSON parser is like "wut" when it sees the // on line 6 of your config file. Removing those lines fixes that error

After removing those lines, I do end up getting an error {"data":"sql: no rows in result set","status":500}, but this is because there aren't any blocks yet in the memento db. After I mined 10 blocks (because you have the lag configured), I was able to see something in the UI

Hope this helps!

SalahAdDin commented 4 years ago

@seesemichaelj don't close this yet, please.

I made the changes you said and even so:

time="2020-08-29T16:53:42Z" level=info msg="[eth] starting best block tracker"
time="2020-08-29T16:53:42Z" level=info msg="[eth] setting up websocket connection"
time="2020-08-29T16:53:43Z" level=warning msg="error connecting to server: dial tcp 172.18.0.2:8545: connect: connection refused "
time="2020-08-29T16:53:50Z" level=info msg="[taskmanager] setting up redis connection"
time="2020-08-29T16:53:50Z" level=info msg="[taskmanager] connected to redis successfully"
time="2020-08-29T16:53:50Z" level=info msg="[core] connecting to postgres"
time="2020-08-29T16:53:50Z" level=fatal msg="[core] dial tcp 172.18.0.3:5432: connect: connection refused"
time="2020-08-29T16:53:53Z" level=info msg="[eth] starting best block tracker"
time="2020-08-29T16:53:53Z" level=info msg="[eth] setting up websocket connection"
time="2020-08-29T16:53:53Z" level=info msg="[taskmanager] setting up redis connection"
time="2020-08-29T16:53:53Z" level=info msg="[taskmanager] connected to redis successfully"
time="2020-08-29T16:53:53Z" level=info msg="[core] connecting to postgres"
time="2020-08-29T16:53:53Z" level=info msg="[core] attempting automatic execution of migrations"
2020/08/29 16:53:53 OK    00001_create_table_blocks.go
2020/08/29 16:53:53 OK    00002_create_table_uncles.go
2020/08/29 16:53:53 OK    00003_create_table_txs.go
2020/08/29 16:53:53 OK    00004_create_table_log_entries.go
2020/08/29 16:53:53 OK    00005_create_table_account_txs.go
2020/08/29 16:53:53 OK    00006_delete_block_function.go
2020/08/29 16:53:53 goose: no migrations to run. current version: 6
time="2020-08-29T16:53:53Z" level=info msg="[core] database version is up to date"
time="2020-08-29T16:53:53Z" level=info msg="[core] connected to postgres successfuly"
time="2020-08-29T16:53:53Z" level=info msg="[core] got highest block from db" block=0
time="2020-08-29T16:53:53Z" level=info msg="[core] got highest block from network" block=0
time="2020-08-29T16:53:53Z" level=info msg="[core] skipping backfilling since feature is disabled"
[GIN] 2020/08/29 - 16:54:37 | 500 |   22.157933ms |      172.18.0.1 | GET      /api/explorer/block/latest
[GIN] 2020/08/29 - 16:54:45 | 200 |   11.596716ms |      172.18.0.1 | GET      /
[GIN] 2020/08/29 - 16:54:45 | 200 |   67.940617ms |      172.18.0.1 | GET      /web/assets/css/index.css
[GIN] 2020/08/29 - 16:54:45 | 200 |   94.735523ms |      172.18.0.1 | GET      /web/assets/css/tailwind-overwrite.css
[GIN] 2020/08/29 - 16:54:45 | 200 |   63.449152ms |      172.18.0.1 | GET      /web/assets/js/progressbar.min.js
[GIN] 2020/08/29 - 16:54:45 | 200 |   63.153452ms |      172.18.0.1 | GET      /web/assets/images/logo.svg
[GIN] 2020/08/29 - 16:54:48 | 200 |     309.287µs |      172.18.0.1 | GET      /web/assets/images/favicon.ico
[GIN] 2020/08/29 - 16:55:27 | 200 |   17.991228ms |      172.18.0.1 | GET      /queue
[GIN] 2020/08/29 - 16:55:32 | 200 |    2.684777ms |      172.18.0.1 | GET      /config
[GIN] 2020/08/29 - 16:55:57 | 200 |    4.410092ms |      172.18.0.1 | GET      /
[GIN] 2020/08/29 - 16:56:10 | 500 |       865.1µs |      172.18.0.1 | GET      /api/explorer/block/latest
time="2020-08-29T16:57:48Z" level=warning msg="[eth] WS connection closed"
time="2020-08-29T16:57:48Z" level=info msg="[eth] setting up websocket connection"
time="2020-08-29T16:57:48Z" level=warning msg="error connecting to server: dial tcp 172.18.0.2:8545: connect: connection refused "
time="2020-08-29T16:57:48Z" level=info msg="[main] Got stop signal. Finishing work."
time="2020-08-29T16:57:48Z" level=info msg="[core] closed best block tracker"
time="2020-08-29T16:57:48Z" level=info msg="[core] closed db connection"
time="2020-08-29T16:57:48Z" level=info msg="[core] closed task manager"
time="2020-08-29T16:57:48Z" level=info msg="[main] Work done. Goodbye!"
time="2020-08-29T16:58:49Z" level=info msg="[eth] starting best block tracker"
time="2020-08-29T16:58:49Z" level=info msg="[eth] setting up websocket connection"
time="2020-08-29T16:58:49Z" level=warning msg="error connecting to server: dial tcp 172.18.0.4:8545: connect: connection refused "
time="2020-08-29T16:58:54Z" level=info msg="[taskmanager] setting up redis connection"
time="2020-08-29T16:58:54Z" level=info msg="[taskmanager] connected to redis successfully"
time="2020-08-29T16:58:54Z" level=info msg="[core] connecting to postgres"
time="2020-08-29T16:58:54Z" level=info msg="[core] attempting automatic execution of migrations"
2020/08/29 16:58:54 goose: no migrations to run. current version: 6
time="2020-08-29T16:58:54Z" level=info msg="[core] database version is up to date"
time="2020-08-29T16:58:54Z" level=info msg="[core] connected to postgres successfuly"
time="2020-08-29T16:58:54Z" level=info msg="[core] got highest block from db" block=0
time="2020-08-29T16:58:54Z" level=info msg="[core] got highest block from network" block=0
time="2020-08-29T16:58:54Z" level=info msg="[core] skipping backfilling since feature is disabled"
[GIN] 2020/08/29 - 16:59:51 | 500 |     388.819µs |      172.18.0.1 | GET      /api/explorer/block/latest

Connection refused by the ganache-cli.

So, i can't see any track in memento and also at explorer.

mikeseese commented 4 years ago

@SalahAdDin Unfortunately this isn't an issue with ganache-cli, so I'm going to keep this issue closed. The issue lies in either your memento or docker configuration. However, I did publish my full configuration at https://github.com/seesemichaelj/memento-docker-compose in hopes that you can figure out what it is that you have configured wrong. This configuration works perfectly for me. Hope it helps!

SalahAdDin commented 4 years ago

@SalahAdDin man, yours does not work for me hahahahaha I cloned it and postgresql gets a problem because it has .gitignore file inside, explorer is not accessible, and memento does not show any track, but ganache is getting transactions, it is something.

mikeseese commented 4 years ago

Ah, truthfully, I didn't test it after I added those .gitignore files 😬

Also, I changed the exposed port for explorer to be 8080, so try going to http://localhost:8080 instead. Binding to port 80 in dev is just a bad idea since some people (i.e. me lol) actually run webservers on their local machines (fun fact: back in the day, even Skype used to bind to port 80 for some stupid reason)

memento may not show any transactions until after 10 blocks. I had to create at least 10 blocks before any of them started getting logged

mikeseese commented 4 years ago

If you wanted to get past the 10 block thing you can disable lag altogether or change the number of blocks here https://github.com/seesemichaelj/memento-docker-compose/blob/master/.volumes/memento/config.yml#L61-L66

SalahAdDin commented 4 years ago

@seesemichaelj i tested your configuration and i got that bug, with your own configuration, i didn't touch it.

mikeseese commented 4 years ago

I don't know what bug you're talking about. If you're talking about the .gitignore files, feel free to remove them before you try running it. All that matters are the configuration files. They work for me, but your machine setup may require changes. Unfortunately we don't support generic Docker or Memento configuration

SalahAdDin commented 4 years ago

THis is really very tricky, now i have a problem with the explorer:

app.bundle.js?34631def42855790e829:477 Couldn't load application config ProgressEvent
(anonymous) @ app.bundle.js?34631def42855790e829:477
/config.json:1 Failed to load resource: net::ERR_CONNECTION_REFUSED

Unfortunately, they are not answering issues in the main repo, It's sad that ganache-cli doesn't offer its own web interface.

trufflesuite / ganache-cli-archive