Azure / DotNetty

DotNetty project – a port of netty, event-driven asynchronous network application framework
Other
4.09k stars 977 forks source link

How Can I increase "DefaultInitial" value of AdaptiveRecvByteBufAllocator #365

Closed liuyugan closed 6 years ago

liuyugan commented 6 years ago

I my case, the client side will send more than 1024 bytes, and then I have a Decoder translate IByteBuffer to byte[] :

 protected override void Decode(IChannelHandlerContext context, IByteBuffer message, List<object> output)
        {
            byte[] sendBytes = new byte[message.ReadableBytes];
            message.ReadBytes(sendBytes);
            output.Add(sendBytes);
        }

since the DefaultInitial was 1024. so the message.Capacity was also 1024 then, I cannot got more bytes from client side.

So, How can I increase this value? Which value should I config it? Further step: How can I set Dotnetty server side Receive package and Send package size?

nayato commented 6 years ago

TCP is a streaming protocol so you might end up getting partial messages regardless of buffer mgmt settings in DotNetty. you have to have framing protocol on top of TCP. For instance, you cloud use length prefix based framing.

liuyugan commented 6 years ago

@nayato Thank you for your quick answer!

Since the the package size was transport by client side, according to our protocol that we have already used, this size cannot be change. now my solution is, set 'Defaultinitial' from 1024 to 1024 4, so my question is : "is any way ,I can set it?" because I think "modify value of Defaultinitial from 1024 up to 1024 4" is not good way.

yyjdelete commented 6 years ago

@liuyugan You should use ByteToMessageDecoder instead of MessageToMessageDecoder<IByteBuffer> to do that. See the below code if you use fixed length.

    public class FixedLengthDecoder : ByteToMessageDecoder
    {
        private readonly int length;

        public FixedLengthDecoder(int length)
        {
            this.length = length;
        }

        protected override void Decode(IChannelHandlerContext context, IByteBuffer input, List<object> output)
        {
            if (input.ReadableBytes < this.length)
                return;

            byte[] sendBytes = new byte[this.length];
            input.ReadBytes(sendBytes);
            output.Add(sendBytes);
        }
    }
StormHub commented 6 years ago

@liuyugan For transport level framing, there are typically two ways of dealing with it

  1. If you know in advance exactly how long the bytes is going to be, you can use LengthFieldPrepender/LengthFieldBasedFrameDecoder as @nayato and @yyjdelete suggested

  2. If the message size varies, you have to have some sort of delimiter in bytes to indicate it is the end of the content. (For example, http protocol indicates end of header by '\r\n\r\n'). you can use LineBasedFrameDecoder

Either way, there is no need to change the AdaptiveRecvByteBufAllocator at all and DotNetty just does everything else automatically for you and you just need to deal with the actual contents :)

liuyugan commented 6 years ago

@yyjdelete ,

Thank you for your response, and I am very sorry for my reply so late.

Yes, your answer was very helpful, however input.Capacity was lower than the size of client send : client send 1026, but input,Capacity was still 1024, so I received bytes was not fully.

So, maybe I didn't explain my question clearly, now hoe can I increase 'input.Capacity'?

yyjdelete commented 6 years ago

@liuyugan Still don't know why you need to increase input.Capacity. Since there is also an input.MaxCapacity exists, the Capacity can auto increase when more data is needed to read. And AdaptiveRecvByteBufAllocator(the default one) will choose the best Capacity next time with the bytes count you read once in the history. If the data you read is uncompleted, the thing you needed is an ByteToMessageDecoder instead.

liuyugan commented 6 years ago

@yyjdelete

Matbe I didn't explane my situation clearly.

Since the package that send by client side is “variable”, example: first time client send 38, and next time it will send 1028, since "Defaultinitial" is 1024 in DotNetty source code, so in the second time I cannot receive message completely.

So, How can i reslove it?

Looking forward your response , Thank you!

yyjdelete commented 6 years ago

@liuyugan You mean split package just by readTimeout? It's not reliable in wide area network due to network lag, socket impl/config and some others, since tcp is transported as Stream, and all packages maybe split/join by routers.

You still need an ByteToMessageDecoder or directly use ChannelHandlerAdapter to do the split, maybe like this(not tested).

code ```cs namespace Test { using DotNetty.Buffers; using DotNetty.Common.Concurrency; using DotNetty.Transport.Channels; using System; using System.Collections.Generic; public class ReadTimeoutFrameDecoder : DotNetty.Codecs.ByteToMessageDecoder { readonly int timeoutMs; IScheduledTask readTimeoutTask; public ReadTimeoutFrameDecoder(int timeoutMs) { this.timeoutMs = timeoutMs; } protected override void Decode(IChannelHandlerContext context, IByteBuffer input, List output) { if (this.timeoutMs > 0) { this.readTimeoutTask?.Cancel(); this.readTimeoutTask = context.Executor.Schedule((ctx, state) => { var self = (ReadTimeoutFrameDecoder)ctx; var ctx2 = (IChannelHandlerContext)state; self.readTimeoutTask = null; self.FireRead(ctx2, true); }, this, context, TimeSpan.FromMilliseconds(this.timeoutMs)); } } public override void ChannelReadComplete(IChannelHandlerContext context) { if (this.timeoutMs <= 0) { this.FireRead(context, false); } base.ChannelReadComplete(context); } public override void ChannelInactive(IChannelHandlerContext ctx) { if (this.timeoutMs > 0) { this?.readTimeoutTask.Cancel(); this.readTimeoutTask = null; } this.FireRead(ctx, true); base.ChannelInactive(ctx); } public override void HandlerRemoved(IChannelHandlerContext context) { if (this.timeoutMs > 0) { this?.readTimeoutTask.Cancel(); this.readTimeoutTask = null; } this.FireRead(context, false); base.HandlerRemoved(context); } private void FireRead(IChannelHandlerContext context, bool fireReadComplete) { var buf = base.InternalBuffer; if (!buf.IsReadable()) return; byte[] sendBytes = new byte[buf.ReadableBytes]; buf.ReadBytes(sendBytes); context.FireChannelRead(sendBytes); if (fireReadComplete) { base.DiscardSomeReadBytes(); context.FireChannelReadComplete(); } } } } ```

And you can also change the default when init the bootstrap if you just want to change the Defaultinitial with bootstrap.Option(ChannelOption.Allocator, new FixedRecvByteBufAllocator(2018))(ChildOption for server)