search

xx-labs / xxstats

Integration of xx network in Polkastats web app
Apache License 2.0
2 stars 1 forks source link

Crawler is not able to fetch metadata at blocks #1697251 and #822557 using susbtrate sidecar #1

Closed mariopino closed 2 years ago

mariopino commented 2 years ago

Crawler is not able to fetch metadata at blocks #1697251 and #822557 using susbtrate sidecar:

{"level":50,"time":1658131160837,"pid":37,"hostname":"d4ec27e807a6","crawler":"blockHarvester","msg":"Error fetching runtime metadata at 0x33e1f5ab29d723d130ae319ab2c270e97007717c4a104ad724e4500b3554b793: {\"message\":\"Request failed with status code 500\",\"name\":\"Error\",\"stack\":\"Error: Request failed with status code 500\\n    at createError (/usr/app/crawler/node_modules/axios/lib/core/createError.js:16:15)\\n    at settle (/usr/app/crawler/node_modules/axios/lib/core/settle.js:17:12)\\n    at IncomingMessage.handleStreamEnd (/usr/app/crawler/node_modules/axios/lib/adapters/http.js:269:11)\\n    at IncomingMessage.emit (node:events:539:35)\\n    at IncomingMessage.emit (node:domain:475:12)\\n    at endReadableNT (node:internal/streams/readable:1345:12)\\n    at processTicksAndRejections (node:internal/process/task_queues:83:21)\",\"config\":{\"url\":\"http://sidecar:9090/runtime/metadata?at=0x33e1f5ab29d723d130ae319ab2c270e97007717c4a104ad724e4500b3554b793\",\"method\":\"get\",\"headers\":{\"Accept\":\"application/json, text/plain, */*\",\"User-Agent\":\"axios/0.21.4\"},\"transformRequest\":[null],\"transformResponse\":[null],\"timeout\":0,\"xsrfCookieName\":\"XSRF-TOKEN\",\"xsrfHeaderName\":\"X-XSRF-TOKEN\",\"maxContentLength\":-1,\"maxBodyLength\":-1,\"transitional\":{\"silentJSONParsing\":true,\"forcedJSONParsing\":true,\"clarifyTimeoutError\":false}}}"}
{"level":50,"time":1658131160841,"pid":37,"hostname":"d4ec27e807a6","crawler":"blockHarvester","msg":"Error adding block #1697251: TypeError: Cannot read properties of undefined (reading 'metadata')"}
{"level":30,"time":1658131160843,"pid":37,"hostname":"d4ec27e807a6","crawler":"blockHarvester","msg":"Processed chunk 1/1 [100.00%] in 1.84s min/max/avg: 1.84/1.84/1.84 cur/avg bps: 5.43/5.43"}
{"level":30,"time":1658131160843,"pid":37,"hostname":"d4ec27e807a6","crawler":"blockHarvester","msg":"Detected gap! Harvesting blocks from #822557 to #822557"}
{"level":30,"time":1658131160843,"pid":37,"hostname":"d4ec27e807a6","crawler":"blockHarvester","msg":"Processing chunks of 10 blocks"}
{"level":30,"time":1658131162107,"pid":37,"hostname":"d4ec27e807a6","crawler":"blockHarvester","msg":"Added block #822557 (0xa8b8…fbc5) in 1.26s"}
{"level":50,"time":1658131162642,"pid":37,"hostname":"d4ec27e807a6","crawler":"blockHarvester","msg":"Error fetching runtime metadata at 0xa8b89fac2366a2463fe0031b3b9774cfdf516d5c89b3ad4b859bcac61b92fbc5: {\"message\":\"Request failed with status code 500\",\"name\":\"Error\",\"stack\":\"Error: Request failed with status code 500\\n    at createError (/usr/app/crawler/node_modules/axios/lib/core/createError.js:16:15)\\n    at settle (/usr/app/crawler/node_modules/axios/lib/core/settle.js:17:12)\\n    at IncomingMessage.handleStreamEnd (/usr/app/crawler/node_modules/axios/lib/adapters/http.js:269:11)\\n    at IncomingMessage.emit (node:events:539:35)\\n    at IncomingMessage.emit (node:domain:475:12)\\n    at endReadableNT (node:internal/streams/readable:1345:12)\\n    at processTicksAndRejections (node:internal/process/task_queues:83:21)\",\"config\":{\"url\":\"http://sidecar:9090/runtime/metadata?at=0xa8b89fac2366a2463fe0031b3b9774cfdf516d5c89b3ad4b859bcac61b92fbc5\",\"method\":\"get\",\"headers\":{\"Accept\":\"application/json, text/plain, */*\",\"User-Agent\":\"axios/0.21.4\"},\"transformRequest\":[null],\"transformResponse\":[null],\"timeout\":0,\"xsrfCookieName\":\"XSRF-TOKEN\",\"xsrfHeaderName\":\"X-XSRF-TOKEN\",\"maxContentLength\":-1,\"maxBodyLength\":-1,\"transitional\":{\"silentJSONParsing\":true,\"forcedJSONParsing\":true,\"clarifyTimeoutError\":false}}}"}
{"level":50,"time":1658131162643,"pid":37,"hostname":"d4ec27e807a6","crawler":"blockHarvester","msg":"Error adding block #822557: TypeError: Cannot read properties of undefined (reading 'metadata')"}

We need to get metadata using polkadot-js api and remove sidecar from docker-compose