Garnet is a remote cache-store from Microsoft Research that offers strong performance (throughput and latency), scalability, storage, recovery, cluster sharding, key migration, and replication features. Garnet can work with existing Redis clients.
I'm trying out the capabilities and limits of Garnet and whether I can replace Redis with it. Using the following PageSize configurations:
{
"PageSize": "128m",
"PubSubPageSize": "128m",
}
I can set keys with fairly large values and I can even publish fairly large pub/sub messages. Except if somebody is subscribed to the channel I am publishing to. Then if the message is longer than 128 * 1024 - 11 (131 061) bytes long, I get the following error:
ERR Garnet Exception: Failed to write to response buffer
In the log of Garnet I see the following:
01::34::57 crit: Session[0] [127.0.0.1:55402] [00EE8998] ProcessMessages threw a GarnetException: Garnet.common.GarnetException: Failed to write to response buffer
at Garnet.common.GarnetException.Throw(String message, LogLevel logLevel)
at Garnet.server.RespServerSession.Publish(Byte*& keyPtr, Int32 keyLength, Byte*& valPtr, Int32 valLength, Byte*& inputPtr, Int32 sid)
at Garnet.server.SubscribeBroker`3.Broadcast(Byte[] key, Byte* valPtr, Int32 valLength, Boolean ascii)
at Garnet.server.RespServerSession.NetworkPUBLISH() at Garnet.server.RespServerSession.ProcessMessages()
at Garnet.server.RespServerSession.TryConsumeMessages(Byte* reqBuffer, Int32 bytesReceived)
Note: I can get rid of this error by compiling an own version of Garnet and increasing the network buffer related limits in NetworkBufferSettings.cs, BufferSizeUtils.cs and MaxSizeSettings.cs. As I do not fully grasp the networking related code of Garnet, I don't know the consequences of changing them to new, fairly large numbers. However I don't think these changes should be required by an end user or we should be able to change these limits via configuration.
Describe the bug
I'm trying out the capabilities and limits of Garnet and whether I can replace Redis with it. Using the following PageSize configurations:
I can set keys with fairly large values and I can even publish fairly large pub/sub messages. Except if somebody is subscribed to the channel I am publishing to. Then if the message is longer than
128 * 1024 - 11
(131 061) bytes long, I get the following error:In the log of Garnet I see the following:
Note: I can get rid of this error by compiling an own version of Garnet and increasing the network buffer related limits in
NetworkBufferSettings.cs
,BufferSizeUtils.cs
andMaxSizeSettings.cs
. As I do not fully grasp the networking related code of Garnet, I don't know the consequences of changing them to new, fairly large numbers. However I don't think these changes should be required by an end user or we should be able to change these limits via configuration.Steps to reproduce the bug
Expected behavior
The publish should work fine with messages allowed by the page size. Redis does not have any problems with these data sizes.
Screenshots
No response
Release version
v1.0.36
IDE
Visual Studio 2022
OS version
Windows 11 Pro
Additional context
No response