search

electric-sql / electric

Sync little subsets of your Postgres data into local apps and services.
https://electric-sql.com
Apache License 2.0
6.47k stars 156 forks source link

Throw error if a shape response doesn't include necessary headers #1950

Closed KyleAMathews closed 3 days ago

KyleAMathews commented 2 weeks ago

Two people on discord today reported they hit an infinite loop where a shape request with offset=-1 came back with a 200 status but was missing the electric-offset header.

Both had proxies that for one reason or another had stripped the header(s).

Electric doesn't work without its headers so we should give a more helpful clue to people writing middleware on what is wrong.

The error message should look something like:

The response for the shape request to ${url} didn't include the following required headers:
- electric-offset
- etc

This is often due to a proxy not allowing all headers to be returned. See this doc page for more information: https://electric-url.com/something
thruflo commented 2 weeks ago

That link at the end of the error message could go to a section on this in the troubleshooting guide.

KyleAMathews commented 2 weeks ago

Yeah that'd be perfect

KyleAMathews commented 2 weeks ago

Perhaps even simpler is we just check that we never make a request to the same URL twice. As that covers a lot of potential problems.

thruflo commented 2 weeks ago

Yup, avoiding the scenarios where we hammer the sever with requests in a loop would be good.

There are cases where we do want to call the same URL more than once — eg when trying to sync a table that doesn’t exist yet, we get an error response and keep trying the same URL until the table is created. So there we maybe want more of a backoff.