RTDB write size limit error

[mortenbekditlevsen]

Step 0: Are you in the right place?

• For issues or feature requests related to the code in this repository file a Github issue.
  • If this is a feature request please use the Feature Request template.
• For general technical questions, post a question on StackOverflow with the firebase tag.
• For general (non-iOS) Firebase discussion, use the firebase-talk google group.
• For backend issues, console issues, and other non-SDK help that does not fall under one of the above categories, reach out to Firebase Support.
• Once you've read this section and determined that your issue is appropriate for this repository, please delete this section.

[REQUIRED] Step 1: Describe your environment

• Xcode version: 11.6
• Firebase SDK version: 6.28.1
• Firebase Component: Database
• Component version: 6.28.1
• Installation method: Carthage

[REQUIRED] Step 2: Describe the problem

Steps to reproduce:

If you call setValue on a DatabaseReference with a value that is larger than the allowed write limit of 16mb (my case was 19mb), the supplied completion handler is never called. Furthermore any subsequent updates from the client are never synced to the server.

I have not created a project to demostrate the issue as I think that it is fairly easy to reproduce. If you wish, I could create a reproduction.

Relevant Code:

let value = <something really big. like a dictionary from a string to a base64-encoded and really large image>
 ref.setValue(value, withCompletionBlock: { error, _ in
   print("Never reached")
})

// sleep for a while
otherRef.setValue(smallValue)

// smallValue never reaches the server

[mortenbekditlevsen]

A bit more info on this. With debug logging enabled and by following along in the code, I can sort of see what is going on - and I can also see that a fix would apparently require a server side change too.

I get the following logs when the write is too large:

2020-10-01 14:12:32.951087+0200 POS[21369:504571] 6.28.0 - [Firebase/Database][I-RDB082011] Got control message: {
    d = "ClientId[9234551]:ErrorId[1002]: Write too large";
    t = e;
}
2020-10-01 14:12:32.951202+0200 POS[21369:504571] 6.28.0 - [Firebase/Database][I-RDB082012] Unknown control message returned from server: {
    d = "ClientId[9234551]:ErrorId[1002]: Write too large";
    t = e;
}

So the first thing here is that the control message type 'e' is unhandled. I looked at the code to see if it might be 'easy' to fix. But then I realise that in order to call the callback, we need to know the request number of the failed write - and the control message from the server apparently does not contain the request number.

Not really certain what to do with this, but perhaps this information can be of use...

[mortenbekditlevsen]

I can see why the server probably can't respond with the request number - as the request is too large and it is likely not parsed at all.

A hacky workaround - even if the server can't respond with the request number - and if you deem the issue worthy to fix: Upon the client receiving a 'Write too large', the client could look at all the outstanding requests, find the largest one, call it's completion with an error and remove it. I know that it is probably a bit expensive to serialize all requests to compare their sizes, but compared to what is already going on (very frequent retrys of all outstanding requests), this work is still not a whole lot.

[schmidt-sebastian]

@mortenbekditlevsen Thanks for filing this. This is actually a known issue for the Web client (https://github.com/firebase/firebase-js-sdk/issues/1158), but probably affects all of our clients. Since only a handful of users seems to be hitting this, we have not yet prioritized a fix. The eventual fix will likely involve us rejecting the write that is too large, which will allow us to process other subsequent writes. If we do prioritize this fix, we will update this issue accordingly.