Trouble with senging files across LAN

[GoogleCodeExporter]

I'm writing client\server app for sending files using your library. When I 
upload one file at time everything is good (most of time). But when I try to 
upload two or more files... here comes trouble. :)

I've included copy of my test project source files and binaries in attachment.

I understand that bug could be in my code instead of yours. If it's that case I 
apologize.

If you have any other questions regarding this issue, please ask. I just do not 
know what information might be useful for you for pinpointing that problem.

Send/Receive buffers is set to 1048576 in both client and server.

Files are saved in system temporary directory (Path.GetTempPath()). After 
processing FinalizeUpload corresponding file is moved to the final location 
(appPath\Uploads).
BeginUpload->UploadChunk (repeat N times)->FinalizeUpload

What steps will reproduce the problem?
0. Compile project in MSVS2010. Or run binaries I've provided in attachment ( 
.net 4.0 is required)
1. Prepare two 30-50Mb files (archives, music, video, whatever. that does not 
matter)
3. Run TestServer.exe
4. Run two instances of TestClient.exe.
5. In "Client1" click "UploadFile" button and select first file.
6. In "Client2" click "UploadFile" button and select second file.
7. Start upload process by cicking Open in dialog boxes.

--- another way
1. Prepare two 50+ Mb files.
2. Run TestServer.exe
3. Run one instance of TestClient.exe
5. Click "UploadFile" button, select file, hit "open" in dialog box. Wait until 
file is uploaded. Close client.
6. Run again TestClient.exe.
7. Try to upload another file.

Sometimes I need to repeat steps, because bugs won't show themselves.

What is the expected output? What do you see instead?
--> I expect that files will be copied to destination (appDir\Uploads). But 
instead I get weird exceptions.

What version of the product are you using? On what operating system?
--> I've included source code of your library with my test project.
--> Windows 7 x64. I've tested my project on Windows XP sp3. Same results.

----- additional information -------

So far I've got three different exceptions:

------- first exception ---------
Source: Lidgren.Network
Stack Trace:
at Lidgren.Network.NetException.Assert(Boolean isOk, String message)
    in \Lidgren.Network\NetException.cs:line 58
at Lidgren.Network.NetIncomingMessage.ReadBytes(Int32 numberOfBytes)
    in \Lidgren.Network\NetIncomingMessage.Read.cs:line 113
at TestServer.Program.ProcessIncomingDataMessage(NetIncomingMessage msg)
    in \TestServer\MessageProcessing.cs:line 24
Target Site: Void Assert(Boolean, System.String)
>Lidgren.Network:Trying to read past the buffer size -
likely caused by mismatching Write/Reads, different size or order.

------- second exception ---------
Source: mscorlib
Stack Trace:
at System.Text.Encoding.ThrowCharsOverflow()
at System.Text.Encoding.ThrowCharsOverflow(DecoderNLS decoder, Boolean 
nothingDecoded)
at System.Text.UTF8Encoding.GetChars(Byte* bytes, Int32 byteCount, Char* chars, 
Int32 charCount, DecoderNLS baseDecoder)
at System.String.CreateStringFromEncoding(Byte* bytes, Int32 byteLength, 
Encoding encoding)
at System.Text.UTF8Encoding.GetString(Byte[] bytes, Int32 index, Int32 count)
at Lidgren.Network.NetIncomingMessage.ReadString()
    in \Lidgren.Network\NetIncomingMessage.Read.cs:line 446
at TestServer.Program.ProcessIncomingDataMessage(NetIncomingMessage msg)
    in \TestServer\MessageProcessing.cs:line 18
Target Site: Void ThrowCharsOverflow()

>mscorlib:The output char buffer is too small to contain the decoded 
characters, encoding 'Unicode (UTF-8)' fallback 
'System.Text.DecoderReplacementFallback'.
Parameter name: chars

------- third exception ---------
Source: mscorlib
Stack Trace:
at System.Buffer.BlockCopy(Array src, Int32 srcOffset, Array dst, Int32 
dstOffset, Int32 count)
at Lidgren.Network.NetBitWriter.ReadBytes(Byte[] fromBuffer, Int32 
numberOfBytes, Int32 readBitOffset, Byte[] destination, Int32 
destinationByteOffset)
    in \Lidgren.Network\NetBitWriter.cs:line 75
at Lidgren.Network.NetIncomingMessage.ReadBytes(Int32 numberOfBytes)
    in \Lidgren.Network\NetIncomingMessage.Read.cs:line 116
at TestServer.Program.ProcessIncomingDataMessage(NetIncomingMessage msg)
    in \TestServer\MessageProcessing.cs:line 24
Target Site: Void BlockCopy(System.Array, Int32, System.Array, Int32, Int32)

>mscorlib:Offset and length were out of bounds for the array or count is 
greater than the number of elements from index to the end of the source 
collection.

Packet parsing error: Offset and length were out of bounds for the array or 
count is greater than the number of elements from index to the end of the 
source collection.

Original issue reported on code.google.com by barlog.t...@gmail.com on 28 Oct 2010 at 3:50

Attachments:

• FileSendTestProject.zip

[GoogleCodeExporter]

I've also been seeing the third exception occurring in my application. I am 
using lidgren version 3.5, and running Windows 7 64 bit. 

Stack Trace:
at System.Buffer.BlockCopy(Array src, Int32 srcOffset, Array dst, Int32 
dstOffset, Int32 count)
at Lidgren.Network.NetBitWriter.ReadBytes(Byte[] fromBuffer, Int32 
numberOfBytes, Int32 readBitOffset, Byte[] destination, Int32 
destinationByteOffset)
    in \Lidgren.Network\NetBitWriter.cs:line 75
at Lidgren.Network.NetIncomingMessage.ReadBytes(Int32 numberOfBytes)
    in \Lidgren.Network\NetIncomingMessage.Read.cs:line 116
at TestServer.Program.ProcessIncomingDataMessage(NetIncomingMessage msg)
    in \TestServer\MessageProcessing.cs:line 24
Target Site: Void BlockCopy(System.Array, Int32, System.Array, Int32, Int32)

>mscorlib:Offset and length were out of bounds for the array or count is 
greater than the number of elements from index to the end of the source 
collection.

Packet parsing error: Offset and length were out of bounds for the array or 
count is greater than the number of elements from index to the end of the 
source collection.

The error occurs regularly when attempting to read to the end of a data 
message, but is not reproducible on demand. It appears to happen when the 
message payload is greater than the MTU, which has been set to 1408.

I've included my code that reads the messages in case I've misused the lidgren 
library. 

case NetIncomingMessageType.Data:
byte msgType = 0;

   msgType = msg.ReadByte();
   byte[] data = new byte[msg.LengthBytes - msg.PositionInBytes];
   msg.ReadBytes(data, 0, msg.LengthBytes - msg.PositionInBytes);
   ret = new NetworkServerMessage((NetworkMessageType)msgType)
   {
      EndPoint = msg.SenderEndpoint,
      Channel = (NetworkChannel)((int)msg.SequenceChannel + (int)msg.DeliveryMethod),
      Message = data
   };
break;

After reading 
http://code.google.com/p/lidgren-network-gen3/issues/detail?id=40, I rewrote 
the code to use bits, however the same exception continued to occur.

case NetIncomingMessageType.Data:
byte msgType = 0;

   msgType = msg.ReadByte();
   int numberOfBytes = (msg.LengthBits + 7) / 8;
   --numberOfBytes; // minus one for msgType
   byte[] data = new byte[numberOfBytes];
   msg.ReadBits(data, 0, msg.LengthBits - (int)msg.Position);

   ret = new NetworkServerMessage((NetworkMessageType)msgType)
   {
      EndPoint = msg.SenderEndpoint,
      Channel = (NetworkChannel)((int)msg.SequenceChannel + (int)msg.DeliveryMethod),
      Message = data
   };
break;

Original comment by s3190...@gmail.com on 10 Feb 2011 at 6:51

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

I encountered this same problem when receiving large messages from multiple 
sources simultaneously. I also was able to fix it. The problem appears to be in 
the NetPeer.Fragmentation code. The variable m_receivedFragmentGroups is 
indexed by the group number, which is part of the header of the received 
message. This does not distinguish between the same group number from different 
senders. For my code, I put in the following (non-optimal) fix by changing 4 
lines. Instead of using an 'int' to index the dictionary, I am now using a 
string, which is a concatenation of the SenderEndpoint and the group number. 
This is using the most recent release from 2011-02-19.

Original Lines:
Lidgren.Network\NetPeer.cs(99):         m_receivedFragmentGroups = new 
Dictionary<int, ReceivedFragmentGroup>();
Lidgren.Network\NetPeer.Fragmentation.cs(18):   private Dictionary<int, 
ReceivedFragmentGroup> m_receivedFragmentGroups;
Lidgren.Network\NetPeer.Fragmentation.cs(112):  if 
(!m_receivedFragmentGroups.TryGetValue(group, out info))
Lidgren.Network\NetPeer.Fragmentation.cs(117):  m_receivedFragmentGroups[group] 
= info;

New lines:
Lidgren.Network\NetPeer.cs(99):         m_receivedFragmentGroups = new 
Dictionary<string, ReceivedFragmentGroup>();
Lidgren.Network\NetPeer.Fragmentation.cs(18):   private Dictionary<string, 
ReceivedFragmentGroup> m_receivedFragmentGroups;
Lidgren.Network\NetPeer.Fragmentation.cs(112):  if 
(!m_receivedFragmentGroups.TryGetValue(im.SenderEndpoint.ToString() + 
group.ToString(), out info))
Lidgren.Network\NetPeer.Fragmentation.cs(117):  m_receivedFragmentGroups[im.Sende
rEndpoint.ToString() + group.ToString()] = info;

Original comment by daniel.d...@gmail.com on 26 Feb 2011 at 12:07

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Excellent find! I've checked in a bug fix in rev 189. It seems to work fine for 
me but I don't have a very good testing case; if you could verify that it works 
well for you it would be awesome.

Original comment by lidg...@gmail.com on 26 Feb 2011 at 9:28

• Changed state: Started
• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Original comment by lidg...@gmail.com on 11 Mar 2011 at 11:17

• Changed state: Fixed
• Added labels: ****
• Removed labels: ****