How to increase the 65K message limit on the receiving side?

[hspaay]

I'm attempting to use go-sse in the hiveot project (iot digital twin hub) and ran into a limitation when pushing messages from the hub to a client via SSE. Everything works well when the messages are smaller than 65K. When they exceed 65K they never arrive. The sending side however does not report any error.

Some digging found that Connection.read() creates parser.New(), which calls bufio.NewScanner() which sets a default buffer of MaxScanTokenSize, which is indeed 65K.

Scanner.Buffer seems to be the way to change this buffer. However I don't see a way to do this in between Connection.read() and the start of scanning.

What is the intended way to change the buffer size after creating a connection?

[tmaxmax]

Hi there and thank you for raising the issue!

You've correctly identified that at the moment there isn't a way to set the internal buffer. I'll push a new version which will add the sse.Connection.Buffer(buf []byte, maxSize int) method to support this.

In the meantime I'm thinking of overhauling the SSE client completely. At least from the issues raised up until this point it seems as if most usage of this library is concerned with the client and I think the experience could be better. Would it be too much to ask for some feedback on the new client proposal? Setting the read buffer would probably be an option on sse.ConnectConfig.

Nevertheless I'll let you know when the update is pushed.

[tmaxmax]

Here it is, please let me know if v0.9.0-pre.2 solves your use case! Feel free to close the issue yourself if everything works.

[hspaay]

Wow that was fast. Better yet, it works like a charm! Thank you so much.

I will take a look at the client proposal. Thanks again!