search

PeerDB-io / peerdb

Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage
https://peerdb.io
Other
2.25k stars 92 forks source link

Mirror info page does not load #2083

Open cvhariharan opened 2 months ago

cvhariharan commented 2 months ago

When trying to pause / edit the mirror, the mirror page does not open. Looking at the requests, there is one request to /api/v1/mirrors/status that returns a 429 with the error

{"code":8, "message":"grpc: received message larger than max (6624263 vs. 4194304)", "details":[]}

Not sure if this related to the issue.

This generally happens after the mirror has been running for a while.

Version: stable-v0.16.2

Restarts also don't fix the issue.

cvhariharan commented 2 months ago

Could be from the grpc limit. https://pkg.go.dev/google.golang.org/grpc#MaxRecvMsgSize

Amogh-Bharadwaj commented 1 month ago

Hi @cvhariharan ! Thanks for flagging ! This is a known issue which would be fixed by end-to-end pagination of our mirror stats which we show in the overview tab when you click on a mirror. As part of the page load in the overview tab we fetch both the mirror status from temporal and also all the batches that we synced - which could be in thousands We are trying to find cycles to implement this in OSS :)