io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Maximum Append Size of 1048576 Exceeded when appending a list of events to a stream

[TomMante]

Hello,

I am currently trying to append to a stream a List of events and I get the following error: INVALID_ARGUMENT: Maximum Append Size of 1048576

In order to append to a stream I am using the method EventStoreDBClient.appendToStream(String streamName, AppendToStreamOptions options, Iterator<EventData> events)

If I provide 1 event at the time, I have no issues, but when I pass a full list of events (in my case a around 500 events (but could be more) this fails with the mentioned error. I tried debugging but I couldn't find the issue.

Is there a difference when appending a list of events instead of 1 event at the time? Any idea where this issue might come from?

System info: Eventstore 22.10.2 (on docker)

• EVENTSTORE_CLUSTER_SIZE=1
• EVENTSTORE_RUN_PROJECTIONS=All
• EVENTSTORE_START_STANDARD_PROJECTIONS=true
• EVENTSTORE_ENABLE_EXTERNAL_TCP=true
• EVENTSTORE_ENABLE_ATOM_PUB_OVER_HTTP=true
• EVENTSTORE_INSECURE=true
• EVENTSTORE_LOG_LEVEL=Verbose

Operating system MacOS Ventura 13.4.1 Java 17 EventstoreDB-Client-Java 4.3.0

Thank you

[YoEight]

Hey @TomMante, it's actually an error message that comes from the server, the default value is indeed 1048576.

You can change its value when starting the server by either using --max-append-size parameter or setting EVENTSTORE_MAX_APPEND_SIZE env variable.

[TomMante]

Hi @YoEight thank you! Indeed the error comes from the server. But my question is this one:

is it correct that if I use the method 'appendToStream' and provide a 500 events iterator, this is sent as a single package that triggers the MAX APPEND SIZE error?

I was expecting it to send 1 message per each event , or in chunks (this is what I ended up doing, sending it in chunks of 50 events). But I guess it is packaged as an optimisation.

Considering that on eventstore.cloud the --max-append-size parameter can not yet be set ( as discussed here), would it be possible/make sense for this library to have another appendToStream method that allows the user to provide a chunk size?

Thank you!

[YoEight]

is it correct that if I use the method 'appendToStream' and provide a 500 events iterator, this is sent as a single package that triggers the MAX APPEND SIZE error?

Yes. However, the max append size error is related to accumulated payload size, so it's possible to trigger it with a single event too.

would it be possible/make sense for this library to have another appendToStream method that allows the user to provide a chunk size?

Technically yes but also consider what I said above, a single event can trigger it. I don't agree on adding such method on the client library itself though. On the Rust client, we have a "client extra" lib with extension methods to augment the core library. We could do the same for the Java and potentially other clients too if there is demand.

[TomMante]

I understand, indeed it wouldn't make sense for a core library, if necessary I could implement this on my side. In the end we found another solution on our side so for us this is not required.

Thank you!