[WIP] Add shelley mainnet features

[xdzurman]

Adds endpoints and adjusts existing functionality to prepare it for incoming shelley hard fork.

Additional guides here and here. Network settings here.

[ksemaev]

Guys, as I understood from https://github.com/vacuumlabs/adalite-backend-service/pull/90#issuecomment-664545705 - you plan to keep on using :latest db-sync/cardano images? They did release a lot of changes, take a look here: https://github.com/input-output-hk/cardano-node/releases/tag/1.18.0 Or by latest you mean the latest releases by them: Cardano Node 1.18.0, Cardano DB Sync 3.0.0 etc? Also If I understood correctly the release a bit different submit-api. The point of their changes is: only those versions will be able to go through shelley update, as they will be compatible with the changes, while current :latest images are byron-only

[xdzurman]

@ksemaev Hello, sorry for the confusion. We did not touch docker compose yet, since there were far too many testnet networks and versions to keep up the pace with on our side - until now, we've been working around db-sync and cardano-rest configs in screen sessions on our server to get things running. Yes, IOHK believes that cardano node 1.18 will be the final version of node. As for db-sync and cardano-rest, these are still being worked on, but we'll most probably use the versions you mentioned. We're still in the development process, but you can track this branch for updates.

I believe we will be able to deliver the changes to docker compose in time and specify the versions explicitly.

[ksemaev]

@xdzurman when I try running from the versions mentioned in https://github.com/vacuumlabs/adalite-backend-service/pull/89/commits/d2af62446debbe9bf4a80f45a15551b2c9c8a11c (Had to to full PG resync from 0) I get this alert:

yarn run v1.15.2
$ /adalite/node_modules/.bin/knex migrate:latest
migration file "20200629223354_create_tx_out_address_index.js" failed
migration failed with error: create index "idx_tx_out_address" on "tx_out" using hash ("address") - relation "idx_tx_out_address" already exists
error: create index "idx_tx_out_address" on "tx_out" using hash ("address") - relation "idx_tx_out_address" already exists
    at Connection.parseE (/adalite/node_modules/pg/lib/connection.js:545:11)
    at Connection.parseMessage (/adalite/node_modules/pg/lib/connection.js:370:19)
    at Socket.<anonymous> (/adalite/node_modules/pg/lib/connection.js:113:22)
    at Socket.emit (events.js:193:13)
    at addChunk (_stream_readable.js:295:12)
    at readableAddChunk (_stream_readable.js:276:11)
    at Socket.Readable.push (_stream_readable.js:231:10)
    at TCP.onStreamRead (internal/stream_base_commons.js:154:17)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Please advice! And I really appreciate the effort you're putting to keep up with the changes by IOHK

[xdzurman]

@ksemaev The migrations we added were added to the db-sync repo in the meantime, I'll remove them.

[ksemaev]

@xdzurman TY very much, please push the changes

[xdzurman]

@ksemaev We're having problems starting cardano-submit-api from docker-compose, did you not have this problem?

[ksemaev]

@xdzurman the recent change resulted yarn run v1.15.2 error Command "knex" not found.

[ksemaev]

Regarding submit api - they changed port, and they now accept connections by default only from localhost. I'm using kubernetes, so config is a bit different, but overall I have

ENTRYPOINT ["/nix/store/7dn6h4ix2y1jc2gg5219mrwgny5fjkjy-cardano-submit-api-exe-cardano-submit-api-2.1.1/bin/cardano-submit-api", \
            "--socket-path", "/node-ipc/node.socket", \
            "--mainnet", \
            "--listen-address", "*", \
            "--config", "/nix/store/fhyvl8jxymk2nrivyv77y1sp432vgz44-submit-api.json" \
            ]

And changed default port to 8090

[vbonini]

Hello. I've been working with @ksemaev and we ran into an issue with the new adalite-shelley.

When attempting to call POST api/v2/txs/history, the application sees this:

 {"code":"Internal","message":"caused by error: relation \"tx_body\" does not exist"}

And the server logs show this:

[2020-07-29T18:19:51.091Z] ERROR: [object Object]/28 on adalite-shelley-69b5cbbf6b-ghf9m: relation "tx_body" does not exist (req.route=/api/v2/txs/history, res.status=500, res.time=8)
    POST /api/v2/txs/history HTTP/1.1

[xdzurman]

@vbonini Hello, tx_body table did not make it through the db-sync updates and it will not be present on shelley, we don't use the endpoints, but I removed the tx_body column from queries and returns for you. Transactions are build very differently to compared to byron and testnet.

[vbonini]

Thank you for that. Now we're getting this error for the same column.

App:

{"code":"Internal","message":"caused by error: column txs.body does not exist"}

Server:

[2020-07-29T21:38:52.592Z] ERROR: [object Object]/29 on adalite-shelley-7d57d87b56-wgbsf: column txs.body does not exist (req.route=/api/v2/txs/history, res.status=500, res.time=7)
    POST /api/v2/txs/history HTTP/1.1

[vbonini]

Hello. After upgrading to cardano-node 1.19.1 and cardano-db-sync 5.0.0 in our setup (as recommended by the Cardano Foundation), we're getting the following error with ADALITE.

    GET /api/account/info/[REDACTED] HTTP/1.1
[2020-09-17T14:51:37.703Z] ERROR: [object Object]/28 on adalite-shelley-7f7db9698-2vbbn: column d.pool_id does not exist (req.route=/api/account/info/:stakeAddress, res.status=500, res.time=5)

It looks like DB-sync had some breaking changes which required us to recreate the DB. Any recommendations on which commit we should be using?

[xdzurman]

@vbonini Hello, we did not update to db-sync 5.0.0 yet as it offered no real benefits for us. They forgot to mention some renames in their changelogs but now that I'm looking into it more in depth, they renamed some columns for some reason. I'll check it on tomorrow and push some changes :) Concretely, they renamed "pool_id" to "pool_hash_id" in the delegation table if you wanted to fix it yourself fast