Recycling outgoing message

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Sending many messages on the connection consumes a lot of memory.
2. Outgoing messages are not recycled.

What is the expected output? What do you see instead?
Memory should stay at some point and not grow inefficiently.

After some profiling I have got to NetConnection.HandleIncommingAcks method and 
I have found that outgoing messages are not recycled.

// TODO: recycle send
http://code.google.com/p/lidgren-network-gen3/source/browse/trunk/Lidgren.Networ
k/NetConnection.Reliability.cs#190

This means that messages are recycled only when the connection is disconnected. 
It is not good for long living connection or for huge traffic connection.

Please add recycling for outgoing messages.

Thanx !

Original issue reported on code.google.com by NN1436...@gmail.com on 5 Sep 2010 at 9:17

[GoogleCodeExporter]

Can you doublecheck your profiling? The TODO comment only regards to recycling 
the NetSending object; not the entire message. This means the NetSending object 
will be garbage collected and a for every send a new NetSending object have to 
be created.
Ie. it does NOT mean that memory should grow continously, it just means the 
garbage collector will have to work a little harder. The NetSending object is 
quite small tho, so it shouldn't make a big difference unless you send a very 
large number of messages.

Long living connection should not have any problems; they're just garbage 
collected and overall memory consumption should be stable over time.

Original comment by lidg...@gmail.com on 5 Sep 2010 at 9:56

• Changed state: Accepted

[GoogleCodeExporter]

I see.

I create a message using NetPeer.CreateMessage and then send it using 
NetPeer.SendMessage .
The messages are sent very fast.

It seems that a data allocated for the message is not freed.
NetSending doesn't take a lot but it takes fair enough to be recycled.

Original comment by NN1436...@gmail.com on 5 Sep 2010 at 10:52

[GoogleCodeExporter]

I'm using .Net Memory Profiler (memprofiler.com) - I haven't been able to track 
any leakage using the DurableServer/Client sample; which uses some different 
delivery methods. How fast is your app growing over a large amount of time?

Original comment by lidg...@gmail.com on 5 Sep 2010 at 11:53

[GoogleCodeExporter]

I have found how to recreate the situation.

Take your samples: ManyServer, ManyClients.

In ManyClients write a lot of data instead of a small message.
http://code.google.com/p/lidgren-network-gen3/source/browse/trunk/Samples/ManyCl
ients/Client.cs

public partial class Client : Form
{
 readonly byte[] SomeData = new byte[10 * 1000 * 1000];

 ...

 #65 om.Write(SomeData);
}

Run ManyServer, Run ManyClients and create a client.
Follow TaskManager and you'll see memory usage increases.
Moreover, after you close the client window, you'll notice that you have 100% 
CPU usage.

Original comment by NN1436...@gmail.com on 6 Sep 2010 at 11:33

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

And set throttle in server to 0 or to some very big value :)

Original comment by NN1436...@gmail.com on 6 Sep 2010 at 11:46

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Now things are clear.

When I send a message it is put to the queue. The data is removed from the 
queue once message is sent.
The problem that I put data too fast to the queue and sine the network is slow, 
the message is queued thus increases memory consumption.

The solution is to not send messages too fast.
But there is no possibility to check how much data is queued per NetConnection.

One possibility is to add a new property to NetConnetionStatistics.
I attach a patch for this.

Original comment by NN1436...@gmail.com on 6 Sep 2010 at 2:51

Attachments:

• unsentdata.patch

[GoogleCodeExporter]

It can be slightly improved with introduction of event that notifies when the 
data can be sent.

NetConfiguration can include MaxUnsentData and event to set.
NetConnection will check the unsentData when sending a message and then raise 
the event when needed.

Original comment by NN1436...@gmail.com on 6 Sep 2010 at 5:00

[GoogleCodeExporter]

The event would be generated on the network thread; which is unlikely to be a 
good place for the user to do stuff.
A property which calculates the amount of unsent data would be a good addition 
tho - it would enable the application to decide for itself if it should add 
more data to the queue.

Original comment by lidg...@gmail.com on 7 Sep 2010 at 6:42

[GoogleCodeExporter]

Added in rev 115; issue considered fixed.

Original comment by lidg...@gmail.com on 7 Sep 2010 at 8:59

• Changed state: Fixed