zeromq / netmq

A 100% native C# implementation of ZeroMQ for .NET
Other
2.94k stars 742 forks source link

Implement similar fix to issue #225 (fixed in #388) for V1Decoder #1009

Open rheatley-pervasid opened 2 years ago

rheatley-pervasid commented 2 years ago

We have seen a customer suffer a crash with the following stack trace

Description: The process was terminated due to an unhandled exception.
Exception Info: System.OverflowException
   at NetMQ.GCBufferPool.Take(Int32)
   at NetMQ.Msg.InitPool(Int32)
   at NetMQ.Core.Transports.V1Decoder.EightByteSizeReady()
   at NetMQ.Core.Transports.V1Decoder.Next()
   at NetMQ.Core.Transports.DecoderBase.Decode(NetMQ.Core.Transports.ByteArraySegment, Int32, Int32 ByRef)
   at NetMQ.Core.Transports.StreamEngine.ProcessInput()
   at NetMQ.Core.Transports.StreamEngine.Handle(Action, System.Net.Sockets.SocketError, Int32)
   at NetMQ.Core.Transports.StreamEngine.FeedAction(Action, System.Net.Sockets.SocketError, Int32)
   at NetMQ.Core.Transports.StreamEngine.InCompleted(System.Net.Sockets.SocketError, Int32)
   at NetMQ.Core.IOObject.InCompleted(System.Net.Sockets.SocketError, Int32)
   at NetMQ.Core.Utils.Proactor.Loop()
   at System.Threading.ThreadHelper.ThreadStart_Context(System.Object)
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
   at System.Threading.ThreadHelper.ThreadStart()

This was on version 4.0.1.6. The crash is in our application which has a RouterSocket bound to "tcp://127.0.0.1:9202"

I tracked down issues #225 and #388 and believe this is the same problem. I have so far failed to reproduce the issue using NetMQ or python zmq, it isn't clear to me how to send a large enough message.

I can provoke the crash with some contrived python code

import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(("127.0.0.1", 9202))
s.send(b'\xFF' * 9 + b'\x00' + b'\x42' * 1000000)

Taking the fix from #388 and applying to V1Decoder.cs prevents the above python code crashing NetMQ. There is a comment in that PR - "Other decoders/encoders are worth checking out too I guess." - I assume this never happened?

Let me know your thoughts

rheatley-pervasid commented 1 year ago

@drewnoakes anything I can do to help progress this?

rheatley-pervasid commented 1 year ago

Oh, looks like it got fixed differently in #1030 Assume you are happy to lose support for messages >2GB and I'll close it?

drewnoakes commented 1 year ago

Apologies for dropping the ball on this.

I'm on my phone and don't have the spec handy. If the type is unsigned then the fix here would be more comprehensive.