search

cardano-foundation / cardano-graphql

GraphQL API for Cardano
Apache License 2.0
259 stars 103 forks source link

cardano-graphql-801-cardano-db-sync can't use snapshot #865

Closed zhy827827 closed 2 months ago

zhy827827 commented 5 months ago

Ask a question

cardano-graphql-8.0.1$ cat start.sh
export NETWORK=mainnet &&\
docker compose pull &&\
RESTORE_SNAPSHOT=https://update-cardano-mainnet.iohk.io/cardano-db-sync/13.2/db-sync-snapshot-schema-13.2-block-10081680-x86_64.tgz \
docker compose up -d &&\
docker compose logs -f

cardano-graphql-8.0.1$ sudo docker logs -f cardano-graphql-801-cardano-db-sync-1
/run/secrets
Generating PGPASS file
Connecting to network specified in configuration.yaml
Downloading snapshot https://update-cardano-mainnet.iohk.io/cardano-db-sync/13.2/db-sync-snapshot-schema-13.2-block-10081680-x86_64.tgz ...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 52.0G  100 52.0G    0     0  95.3M      0  0:09:18  0:09:18 --:--:-- 94.9M
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   121  100   121    0     0    355      0 --:--:-- --:--:-- --:--:--   355

db-sync-snapshot-schema-13.2-block-10081680-x86_64.tgz: OK
db/
db/4128.dat.gz
db/4125.dat.gz
db/4046.dat.gz
db/4127.dat.gz
db/4152.dat.gz
db/4050.dat.gz
db/4109.dat.gz
db/4115.dat.gz
db/4083.dat.gz
db/4062.dat.gz
db/4072.dat.gz
db/4036.dat.gz
db/4130.dat.gz
db/4081.dat.gz
db/4070.dat.gz
db/4087.dat.gz
db/4058.dat.gz
db/4132.dat.gz
db/4068.dat.gz
db/4142.dat.gz
db/4064.dat.gz
db/4123.dat.gz
db/4060.dat.gz
db/4093.dat.gz
db/4097.dat.gz
db/4040.dat.gz
db/4113.dat.gz
db/4042.dat.gz
db/4038.dat.gz
db/4078.dat.gz
db/4150.dat.gz
db/4085.dat.gz
db/4111.dat.gz
db/4091.dat.gz
db/4105.dat.gz
db/4134.dat.gz
db/4119.dat.gz
db/4144.dat.gz
db/4138.dat.gz
db/toc.dat
db/4089.dat.gz
db/4066.dat.gz
db/4101.dat.gz
db/4156.dat.gz
db/4140.dat.gz
db/4079.dat.gz
db/4044.dat.gz
db/4136.dat.gz
db/4099.dat.gz
db/4117.dat.gz
db/4146.dat.gz
db/4048.dat.gz
db/4154.dat.gz
db/4054.dat.gz
db/4056.dat.gz
db/4052.dat.gz
db/4121.dat.gz
db/4076.dat.gz
db/4074.dat.gz
db/4034.dat.gz
db/4148.dat.gz
db/4107.dat.gz
db/4103.dat.gz
db/4095.dat.gz
119404784-058c10851a-473.lstate.gz
/nix/store/1bys3yjgm97yxy91sfy8f5law41fyif9-postgresql-setup.sh: line 194: sed: command not found

I copy sed from the host to the container, but there is still a problem bash: /bin/sed: cannot execute: required file not found

version: cardano-graphql-8.0.1 ghcr.io/intersectmbo/cardano-db-sync:13.2.0.1 cardanosolutions/cardano-node-ogmios:v6.1.0_8.7.3-mainnet

how to Fix this problem?

Kammerlo commented 5 months ago

This seems like a db-sync issue, I'll discuss it with these devs and come back to you as soon as possible.

Kammerlo commented 4 months ago

@zhy827827 Just to let you know the ticket is open here at db-sync. https://github.com/IntersectMBO/cardano-db-sync/pull/1662

vino-evan commented 4 months ago

I encountered the same problem, how to solve it, version 8.0.3

Kammerlo commented 3 months ago

It seems, it's not included in the current release: https://github.com/IntersectMBO/cardano-db-sync/commit/9f4653ed28b4e44b9c5e9f8fca3b7f380e152eab#commitcomment-141727357

I'll track this issue and keep you updated.

vino-evan commented 3 months ago

You can use this version of the docker image：

docker pull ghcr.io/intersectmbo/cardano-db-sync:13.2.0.2-docker
Kammerlo commented 3 months ago

This version is also used in 8.0.3 of graphQL. Does ist solve the issue for you?

zhy827827 commented 3 months ago

hello @Kammerlo , i used 8.0.3 of graphQL,solve the issue,Thank you!

But there is another issues: {"name":"cardano-graphql","hostname":"cf35e9133cb7","pid":1,"level":30,"module":"Server","msg":"Sync Progress: cardano-db-sync: 100% | Asset: 1%","time":"2024-05-20T03:32:38.288Z","v":0}

The Asset calculation was always 1% , unchanged

Kammerlo commented 3 months ago

Are there any other log outputs in another service? How do you start it?

zhy827827 commented 3 months ago

I used the startup script

export NETWORK=mainnet &&\
docker compose pull &&\
#RESTORE_SNAPSHOT=https://update-cardano-mainnet.iohk.io/cardano-db-sync/13.2/db-sync-snapshot-schema-13.2-block-10272678-x86_64.tgz \
docker compose up -d &&\
docker compose logs -f

Not using snapshots, synchronizing from Genesis block.There are no other error log here.

docker logs -f cardano-graphql-803-server-1 --tail 100

{"name":"cardano-graphql","hostname":"cf35e9133cb7","pid":1,"level":30,"module":"Server","msg":"Sync Progress: cardano-db-sync: 100% | Asset: 1%","time":"2024-05-20T07:52:53.248Z","v":0}
{"name":"cardano-graphql","hostname":"cf35e9133cb7","pid":1,"level":30,"module":"CardanoNodeClient","msg":"initialized","time":"2024-05-20T07:52:53.250Z","v":0}
{"name":"cardano-graphql","hostname":"cf35e9133cb7","pid":1,"level":30,"module":"CardanoNodeClient","tip":{"slot":124625243,"id":"0ad8c996c4d3efde8d6b75199779b6d17ef16aa37fc8a67ac83c6982f01a752c"},"msg":"[Object: null prototype] {\n  slot: 124625243,\n  id: '0ad8c996c4d3efde8d6b75199779b6d17ef16aa37fc8a67ac83c6982f01a752c'\n}","time":"2024-05-20T07:52:53.256Z","v":0}
{"name":"cardano-graphql","hostname":"cf35e9133cb7","pid":1,"level":30,"module":"Server","msg":"Sync Progress: cardano-db-sync: 100% | Asset: 1%","time":"2024-05-20T07:53:08.979Z","v":0}
{"name":"cardano-graphql","hostname":"cf35e9133cb7","pid":1,"level":30,"module":"CardanoNodeClient","msg":"initialized","time":"2024-05-20T07:53:08.981Z","v":0}
{"name":"cardano-graphql","hostname":"cf35e9133cb7","pid":1,"level":30,"module":"CardanoNodeClient","tip":{"slot":124625243,"id":"0ad8c996c4d3efde8d6b75199779b6d17ef16aa37fc8a67ac83c6982f01a752c"},"msg":"[Object: null prototype] {\n  slot: 124625243,\n  id: '0ad8c996c4d3efde8d6b75199779b6d17ef16aa37fc8a67ac83c6982f01a752c'\n}","time":"2024-05-20T07:53:08.988Z","v":0}
{"name":"cardano-graphql","hostname":"cf35e9133cb7","pid":1,"level":30,"module":"Server","msg":"Sync Progress: cardano-db-sync: 100% | Asset: 1%","time":"2024-05-20T07:53:24.852Z","v":0}
{"name":"cardano-graphql","hostname":"cf35e9133cb7","pid":1,"level":30,"module":"CardanoNodeClient","msg":"initialized","time":"2024-05-20T07:53:24.854Z","v":0}
{"name":"cardano-graphql","hostname":"cf35e9133cb7","pid":1,"level":30,"module":"CardanoNodeClient","tip":{"slot":124625306,"id":"6ac6c01e49d3af454d88437d072158906142c894e833ee0a1ce8f7f43abb0b55"},"msg":"[Object: null prototype] {\n  slot: 124625306,\n  id: '6ac6c01e49d3af454d88437d072158906142c894e833ee0a1ce8f7f43abb0b55'\n}","time":"2024-05-20T07:53:24.862Z","v":0}
{"name":"cardano-graphql","hostname":"cf35e9133cb7","pid":1,"level":30,"module":"Server","msg":"Sync Progress: cardano-db-sync: 100% | Asset: 1%","time":"2024-05-20T07:53:40.693Z","v":0}
{"name":"cardano-graphql","hostname":"cf35e9133cb7","pid":1,"level":30,"module":"CardanoNodeClient","msg":"initialized","time":"2024-05-20T07:53:40.695Z","v":0}
{"name":"cardano-graphql","hostname":"cf35e9133cb7","pid":1,"level":30,"module":"CardanoNodeClient","tip":{"slot":124625306,"id":"6ac6c01e49d3af454d88437d072158906142c894e833ee0a1ce8f7f43abb0b55"},"msg":"[Object: null prototype] {\n  slot: 124625306,\n  id: '6ac6c01e49d3af454d88437d072158906142c894e833ee0a1ce8f7f43abb0b55'\n}","time":"2024-05-20T07:53:40.698Z","v":0}

docker logs -f cardano-graphql-803-background-1 --tail 100

{"name":"background","hostname":"a4706ee2e95a","pid":1,"level":30,"module":"HasuraClient","assetId":"a80b9489e5a8e5e5eadcc0c2856f46c5883f5c3d88676ce5559d8cdc","msg":"Adding metadata to asset","time":"2024-05-20T07:58:03.975Z","v":0}
{"name":"background","hostname":"a4706ee2e95a","pid":1,"level":30,"module":"HasuraClient","assetId":"bb9cf64bfca2c8611aefb81b5341ec5f7dba39ec2b6464a4e8a13e51","msg":"Adding metadata to asset","time":"2024-05-20T07:58:03.980Z","v":0}
{"name":"background","hostname":"a4706ee2e95a","pid":1,"level":30,"module":"HasuraClient","assetId":"f37e16045a5aa79503c1e3a36c0892a04a6f4471e4a097c9c98d3854","msg":"Adding metadata to asset","time":"2024-05-20T07:58:03.985Z","v":0}
{"name":"background","hostname":"a4706ee2e95a","pid":1,"level":30,"module":"HasuraClient","assetId":"aebcb6eaba17dea962008a9d693e39a3160b02b5b89b1c83e537c599","msg":"Adding metadata to asset","time":"2024-05-20T07:58:03.990Z","v":0}
{"name":"background","hostname":"a4706ee2e95a","pid":1,"level":30,"module":"HasuraClient","assetId":"8cf66d8c74af3aa53ba498b0d3f18a21f7ed3d12128850bb2b61146d","msg":"Adding metadata to asset","time":"2024-05-20T08:00:08.382Z","v":0}
{"name":"background","hostname":"a4706ee2e95a","pid":1,"level":30,"module":"HasuraClient","assetId":"dc1c91e3afb2f7ad8a837eb0fe4c75f7250f968d55f4e426e4ec2ff4","msg":"Adding metadata to asset","time":"2024-05-20T08:00:13.622Z","v":0}
{"name":"background","hostname":"a4706ee2e95a","pid":1,"level":30,"module":"HasuraClient","assetId":"8cf66d8c74af3aa53ba498b0d3f18a21f7ed3d12128850bb2b61146d","msg":"Adding metadata to asset","time":"2024-05-20T08:00:13.635Z","v":0}
{"name":"background","hostname":"a4706ee2e95a","pid":1,"level":30,"module":"HasuraClient","assetId":"dc1c91e3afb2f7ad8a837eb0fe4c75f7250f968d55f4e426e4ec2ff4","msg":"Adding metadata to asset","time":"2024-05-20T08:00:13.639Z","v":0}
curl   -X POST   -H "Content-Type: application/json"   -d '{"query": "{ cardanoDbMeta { initialized syncPercentage }}"}'   http://localhost:3200/graphql |jq
{
  "data": {
    "cardanoDbMeta": {
      "initialized": true,
      "syncPercentage": 100
    }
  }
}
Kammerlo commented 3 months ago

For how long did you wait regarding the Assets? Because sometimes it can take sometime until the percentage rises further, due to syncing speed.

zhy827827 commented 3 months ago

I have been waiting for three days now

Kammerlo commented 3 months ago

I'll check it

zhy827827 commented 3 months ago

hello @Kammerlo
Is there any progress on this matter?Thank you for your contribution to the community

Kammerlo commented 3 months ago

I'm still on it. Did your node reach the tip?

zhy827827 commented 3 months ago

yes.new tip slot Is normally synchronized

Kammerlo commented 3 months ago

fyi, I'm currently doing some tests. An issue could be the limit of requests in direction of the tokenregistry. Next week I'm planning to change the way we are fetching assets from graphql, then this won't be an issue anymore.

Kammerlo commented 3 months ago

Hey @zhy827827 I tried it on my end on mainnet. I ended up with Assets at 100%, even while the node didn't reach the tip yet. I even added extra parsing safety for Blocks in chainfollower (which is responsible to fill the asset table). Could you try again with this branch https://github.com/cardano-foundation/cardano-graphql/pull/872?

If it doesn't help I would propose two options to get closer to solve your problem:

  1. Try a resync with the new branch
  2. We can have a call and look into the issue together, if you like
zhy827827 commented 3 months ago

Thank you very much.My Asset are still in sync at 1%。 I will try again using the new branch.But I want to know how you deployment graphql.

Here are all the images I have used:

614f25b19a53   cardanofoundation/cardano-graphql-background:8.0.1-mainnet   "node background.js"     7 days ago    Up 3 hours                                                                                        cardano-graphql-803-background-1
99bbdb814597   cardanofoundation/cardano-graphql-hasura:8.0.1               "docker-entrypoint.s…"   7 days ago    Up 7 days             0.0.0.0:8099->8080/tcp, :::8099->8080/tcp                                   cardano-graphql-803-hasura-1
0899d44d2b9d   ghcr.io/intersectmbo/cardano-db-sync:13.2.0.2                "/nix/store/1fk383dn…"   7 days ago    Up 3 hours                                                                                        cardano-graphql-803-cardano-db-sync-1
323dd0be8dcc   postgres:14.10-alpine                                        "docker-entrypoint.s…"   7 days ago    Up 7 days             0.0.0.0:5432->5432/tcp, :::5432->5432/tcp                                   cardano-graphql-803-postgres-1
a049eacec6ae   cardanofoundation/cardano-graphql-server:8.0.1-mainnet       "node index.js"          7 days ago    Up 6 days             3200/tcp, 0.0.0.0:3200->3100/tcp, :::3200->3100/tcp                         cardano-graphql-803-server-1
09142ee1b887   cardanosolutions/cardano-node-ogmios:v6.2.0_8.9.0-mainnet    "/tini -g -- /root/c…"   7 days ago    Up 7 days (healthy)   3000/tcp, 12788/tcp, 0.0.0.0:1337->1337/tcp, :::1337->1337/tcp, 12798/tcp   cardano-graphql-803-cardano-node-ogmios-1
Kammerlo commented 3 months ago

I just started it with sudo docker compose up -d --build Without passing anything regarding the network mainnet is chosen as a default.

Kammerlo commented 2 months ago

Closing this issue due to inactivity. If you have any other issues @zhy827827 let me know or reopen this issue