snarfed / bridgy-fed

🌉 A bridge between decentralized social network protocols
https://fed.brid.gy
Creative Commons Zero v1.0 Universal
767 stars 39 forks source link

Error handling for `Object`s that fail because they're too big #636

Open snarfed opened 1 year ago

snarfed commented 1 year ago

Started seeing this error yesterday on https://maurice-renck.de/de/blog/2023/podcaster-plugin-3 (parsed mf2), which includes a big inlined data: img. Datastore raised google.api_core.exceptions.InvalidArgument: 400 entity is too big, which we don't yet handle, so we 500ed.

Datastore max property value size and entity size are both 1MB. OK. Total parsed mf2 size is only 200KB though, and the converted as1 computed property is similarly ~200KB. So why does the datastore think we're over 1MB?

...arguably it doesn't matter, we should just handle this exception and return HTTP 400.

snarfed commented 2 weeks ago

1501 is probably the most common cause of this, doing that should help here.