The metadata service is stuck [about to start syncing]

[YuXiaoCoder]

Summary

Hi, I am running a Cardano node on the mainnet with the following versions:

ghcr.io/intersectmbo/cardano-node:9.1.0
cardanosolutions/ogmios:v6.5.0
postgres:16.3-alpine
cardanofoundation/cardano-graphql-server:8.2.0-mainnet
cardanofoundation/cardano-graphql-background:8.2.0-mainnet
ghcr.io/intersectmbo/cardano-db-sync:13.3.0.0
cardanofoundation/cardano-graphql-hasura:8.2.0
cardanofoundation/cf-token-metadata-registry-api:1.0.2

Due to the database upgrade, I reloaded the latest db-sync snapshot and cardano-node snapshot

# cardano-node
wget --no-check-certificate https://downloads.csnapshots.io/mainnet/$(curl -s -k https://downloads.csnapshots.io/mainnet/mainnet-db-snapshot.json | grep -o 'mainnet-db-[0-9]*.tar.lz4')

# db-sync
https://update-cardano-mainnet.iohk.io/cardano-db-sync/13.3/db-sync-snapshot-schema-13.3-block-10611621-x86_64.tgz

Steps to reproduce the bug

After the service starts, everything works fine, but cardano-graphql-server is not populated with Asset data, which prevents the interface from being used After the service starts, everything works fine, but metadata 一直显示即将同步数据，cardano-graphql-server 显示没有Remote schema is missing Block

metadata

2024-08-09T02:16:51.653Z  INFO 4324 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port 8080 (http) with context path ''
2024-08-09T02:16:51.668Z  INFO 4324 --- [           main] c.t.r.a.TokenMetadataRegistryApplication : Started TokenMetadataRegistryApplication in 6.362 seconds (process running for 7.079)
2024-08-09T02:16:52.273Z  INFO 4324 --- [nio-8080-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring DispatcherServlet 'dispatcherServlet'
2024-08-09T02:16:52.274Z  INFO 4324 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet        : Initializing Servlet 'dispatcherServlet'
2024-08-09T02:16:52.275Z  INFO 4324 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet        : Completed initialization in 1 ms
2024-08-09T02:17:51.667Z  INFO 4324 --- [   scheduling-1] o.c.t.r.s.TokenMetadataSyncCronJob       : about to start syncing

kubectl logs ada-1-0 ada-1-cardano-graphql-server -f

[2024-08-09 02:35:10] [entrypoint.sh] exec command [node index.js]
{"name":"cardano-graphql","hostname":"ada-1-0","pid":23,"level":30,"module":"CardanoNodeClient","msg":"Initializing. This can take a few minutes...","time":"2024-08-09T02:35:11.712Z","v":0}
{"name":"cardano-graphql","hostname":"ada-1-0","pid":23,"level":30,"module":"CardanoNodeClient","msg":"[Object: null prototype] {\n  slot: 131604591,\n  id: '3392f5c50bd82e3fe3251603a530828672a541828b663c49b1078c54c8266f35'\n}","time":"2024-08-09T02:35:11.737Z","v":0}
{"name":"cardano-graphql","hostname":"ada-1-0","pid":23,"level":30,"module":"CardanoNodeClient","msg":"Initialized","time":"2024-08-09T02:35:11.738Z","v":0}
{"name":"cardano-graphql","hostname":"ada-1-0","pid":23,"level":30,"module":"Server","msg":"Initializing","time":"2024-08-09T02:35:11.738Z","v":0}
{"name":"cardano-graphql","hostname":"ada-1-0","pid":23,"level":30,"module":"HasuraClient","msg":"Initializing","time":"2024-08-09T02:35:11.933Z","v":0}
{"name":"cardano-graphql","hostname":"ada-1-0","pid":23,"level":50,"msg":"Remote schema is missing Block","time":"2024-08-09T02:38:35.925Z","v":0}

Actual Result

cardano-graphql does not work

Expected Result

cardano-graphql is up and running

Environment

{
  os: {
    arch: 'x64',
    cpus: {
      count: 32,
      model: 'Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz'
    },
    release: '4.19.91-24.1.al7.x86_64',
    totalMem: 265924902912,
    type: 'Linux'
  },
  version: '8.0.0'
}

Platform

• [X] Linux (Ubuntu)
• [ ] Linux (Other)
• [ ] macOS
• [ ] Windows

Platform version

Ubuntu

Runtime

• [X] Node.js
• [X] Docker

Runtime version

v18.20.3

[Kammerlo]

Hello, so two things: First of all the tokenregistry takes some time to start, when this is done the background service will start syncing the assets. And did you set these environment variables to the background service as stated within the Readme? CHAIN_FOLLOWER_START_SLOT=23068800 \ CHAIN_FOLLOWER_START_ID=a650a3f398ba4a9427ec8c293e9f7156d81fd2f7ca849014d8d2c1156c359b3a

This is needed otherwise the background service will get stuck.

[YuXiaoCoder]

I was referring to this docker-compose link, where CHAIN_FOLLOWER_START_SLOT and CHAIN_FOLLOWER_START_ID are variables that are set for background containers, not token-metadata-registry

[Kammerlo]

I tried to recreate this issue in my environment without success. Could you elaborate further how your environment looks like? Could you have a look into the db-sync logs and see if there's a problem? An issue could be that the used db-sync version has a known issue which hinders using snapshots. I'm currently working on upgrading db-sync within GraphQL, since the update was just released two days ago.

[YuXiaoCoder]

I used the latest version of db-sync and tried again

[niso1985]

I will share my log of my successes.

2024-08-14T18:18:19.426Z  INFO 11 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet        : Completed initialization in 1 ms
2024-08-14T18:19:18.275Z  INFO 11 --- [   scheduling-1] o.c.t.r.s.TokenMetadataSyncCronJob       : about to start syncing
2024-08-14T18:19:18.275Z  INFO 11 --- [   scheduling-1] o.c.t.registry.service.GitService        : exists and is git repo
d2024-08-14T20:05:19.795Z  INFO 11 --- [   scheduling-1] o.c.t.r.s.TokenMetadataSyncCronJob       : syncing completed

[linconvidal]

Hey @YuXiaoCoder! Have you found a solution to that? If not, could you please share your db-sync logs?

[YuXiaoCoder]

Hey @YuXiaoCoder! Have you found a solution to that? If not, could you please share your db-sync logs?

After updating the version, it's already working, thanks