About 15 records limit.

[GoogleCodeExporter]

What steps will reproduce the problem?
1. When the proto file is complex like:
message A{
  message B{
   ...
  }
  message C{
   ...
  }
  required B myB=1;
  repeated C myC=15;
}
2.
3.

What is the expected output? What do you see instead?
  This protocol is designed for Internet usage. We want that server and client using this proto file can connect to eache other correctly.The Server side using C++ of ProtoBuf; The Client side use this protobuf-csharp-port to generate source code.

   As pointed, when C part of message A has been responsed a 14 items response to the client, all things can parsed correctly; But when the server had a response more then(or equal to) 15 items of C part, It will report error information on client:
"""""
     Unhandled Exception: Google.ProtocolBuffers.InvalidProtocolBufferException: Protocol message contained an invalid tag (zero).
  at Google.ProtocolBuffers.CodedInputStream.ReadTag () [0x00000] in <filename unknown>:0 
  at LX_RESP_ENVELOPE+Types+LX_RESP+Builder.MergeFrom (Google.ProtocolBuffers.CodedInputStream input, Google.ProtocolBuffers.ExtensionRegistry extensionRegistry) [0x00000] in <filename unknown>:0 
  at Google.ProtocolBuffers.AbstractBuilder`2[TMessage,TBuilder].WeakMergeFrom (Google.ProtocolBuffers.CodedInputStream input, Google.ProtocolBuffers.ExtensionRegistry registry) [0x00000] in <filename unknown>:0 
  at Google.ProtocolBuffers.CodedInputStream.ReadMessage (IBuilder builder, Google.ProtocolBuffers.ExtensionRegistry extensionRegistry) [0x00000] in <filename unknown>:0 
  at LX_RESP_ENVELOPE+Builder.MergeFrom (Google.ProtocolBuffers.CodedInputStream input, Google.ProtocolBuffers.ExtensionRegistry extensionRegistry) [0x00000] in <filename unknown>:0 
  at LX_RESP_ENVELOPE+Builder.MergeFrom (Google.ProtocolBuffers.CodedInputStream input) [0x00000] in <filename unknown>:0 
  at Google.ProtocolBuffers.AbstractBuilder`2[TMessage,TBuilder].MergeFrom (System.Byte[] data) [0x00000] in <filename unknown>:0 
  at LX_RESP_ENVELOPE.ParseFrom (System.Byte[] data) [0x00000] in <filename unknown>:0 
"""""
   where LX_RESP_ENVELOPE is like the "message A" mentioned above.

What version of the product are you using? On what operating system?
We are using the protobuf-csharp-port V0.9.1; On Linux operation, Mono based.

Please provide any additional information below.
for the server response information ,the protbuf-csharp-prot can only process 
14 repeated sub messages; we had proved that all bytes send from server had 
been received by the client using protobuf-csharp-port.

Original issue reported on code.google.com by qhlonl...@163.com on 25 Apr 2011 at 11:52

[GoogleCodeExporter]

It should be absolutely fine - it looks like you've probably got an invalid 
file, or you're reading it badly.

If you could show the code that you're using to write on the C++ side and read 
on the C# side, we may be able to help more.

Original comment by jonathan.skeet on 25 Apr 2011 at 12:18

[GoogleCodeExporter]

Ok, Here Is the Proto file between "--------" lines:
-----------------------------------------------------
message REQ_GETTS 
{
    required string     REQID = 1;
    optional string     REQMD5 = 2;
}

message REQ_NEWT
{
    required string     REQID = 1;
    required bytes      REQURL = 2;
    optional bytes      COOKIE = 3;
}

message REQ_DELT
{
    required string     REQID = 1;
    required string     TASKID = 2;
}

message REQ_SAVE
{
    required string     REQID = 1;
    required string     TASKID = 2;
}

message REQ_RENAME
{
    required string     REQID = 1;
    required string     TASKID = 2;
}

message LX_REQ_ENVELOPE
{
    //request header
    message UCLX_REQ
    {
        required int32              COMMAND = 1;
        optional REQ_GETS       GET_T = 2;
        optional REQ_NEWT       NEW_T = 3;
        optional REQ_DELT       DEL_T = 4;
        optional REQ_SAVE       SAVE_DISK = 5;
        optional REQ_RENAME     RENAME = 6;
    }

    message LX_CLIENT_INFO
    {
        required int OP = 1;
        required int JOP = 2;
        required string PFI = 3;
        required string VER = 4;
        required string BSOE = 5;
        optional string IMSS = 6;
        optional string FREE = 7;
        optional string PRDD = 8;
                optional int NEWAPP  =9;
                optional string ANAME =10;
                optional string ATYPE =11;
                optional string ALONG =12;
    }

    required  LX_CLIENT_INFO NEW_CLIENT = 1;

    repeated LX_REQ NEW_LX_REQ = 15;
}

//response envelope
message LX_RESP_ENVELOPE
{
    message LX_RESP
    {
        required int32      COMM = 1;
        required int32      RES = 2;
        required string     REQID = 3;
        optional bytes      LOGURL = 4;
    }

    //response message
    message IN_COMMRESP
    {
        message INT_INFO
        {
            required string TASKID = 1;
            optional int32  RESP_STAT = 2;
            optional int32  ECODE = 3;
            optional bytes  EDESC = 4;
            optional bytes  VAR1 = 5;
            optional bytes  VAR2 = 6;
            optional bytes  VAR3 = 7;
            optional bytes  VAR4 = 8;
                        optional int    VAR5 = 9;
                        optional int    VAR6 =10;
                        optional bytes  VAR7 =11;
                        optional bytes  VAR8 =12;
                        optional int    VARSIZE=13;
        }

        message INT_EXINFO
        {
            optional bytes  NDDESC = 1;
            optional bytes  NDADD = 2;
        }

        repeated INT_INFO   INFOLIST = 1;
        optional string     INFOMD5 = 2;
        optional INT_EXINFO EXINFO = 3;
        optional string     EXMD5 = 4;
    }

    optional IN_COMMRESP COMMRESP = 1;

    repeated LX_RESP ENV_LX_RESP = 15;
}
-----------------------------------------------------

What we get from response, are "INT_INFO" data of "IN_COMMRESP" message, they 
are repetead, And we can mostly parse out 14 of them as said before.

Regards! 

Original comment by qhlonl...@163.com on 26 Apr 2011 at 3:20

[GoogleCodeExporter]

That's given the proto file, but none of the code either reading or writing the 
data... which is where I'm sure the problem is.

If you could write a short but *complete* program demonstrating the problem, 
that would make it a lot easier to track it down.

Original comment by jonathan.skeet on 26 Apr 2011 at 5:22

[GoogleCodeExporter]

I had writen a C# Server side which give response data:
public class RequestPackage{
private global::LX_RESP_ENVELOPE.Builder requestpackage;
  public RequestPackage(){
      requestpackage = global::LX_RESP_ENVELOPE.CreateBuilder();
  }
  public void SetClientInfo(  int op,      
                              int jop,    
                              string pfi,  
                              string ver,
                              string bsoe)
  global::LX_REQ_ENVELOPE.Types.LX_CLIENT_INFO.Builder clientinfo =
           global::LX_REQ_ENVELOPE.Types.LX_CLIENT_INFO.CreateBuilder();
  clientinfo.SetOP(op);
  clientinfo.SetJOP(op);
  if (null == pfi)  clientinfo.SetPFI(""); else clientinfo.SetPFI(pfi);
  if (null == ver)  clientinfo.SetVER(""); else clientinfo.SetVER(ver);
  if (null == bsoe) clientinfo.SetBSOE(""); else clientinfo.SetBSOE(bsoe); 
  requestpackage.SetNEWCLIENT(clientinfo);
  }

  public bool AddGetTasksRequest  (string REQID,  
                                   string REQMD5){
      global::LX_REQ_ENVELOPE.Types.LX_REQ.Builder gettasks_req =
          global::LX_REQ_ENVELOPE.Types.LX_REQ.CreateBuilder();
      global::LX_GETTS.Builder gettask_command = global::LX_GETTS.CreateBuilder();
      if(REQID == null){
          return false; //This Is Required!
      }
      gettask_command.SetREQID(REQID);
      if(null != REQMD5)  gettask_command.SetREQMD5(REQMD5);
      gettasks_req.SetCOMMAND(1); //GETTASKS COMMAND
      gettasks_req.SetGET_T(gettask_command);
      requestpackage.AddRPTUCLXREQ(gettasks_req);            
  }
}
//////////////////////////////Eg.  This Is Require Part. Use 
responsepackage.Build().ToByteArray() to get the ProtoBuf encodede request 
package; 

//////////////////////////// The Response side, use ProtoBuf/C++, as follows:
  void Test:pack(...)
{
 LX_RESP_ENVELOPE resp;
 MakeData(0, resp);
}
void Test::MakeData(int command, LX_RESP_ENVELOPE& lx_resp)
{
 LX_RESP_ENVELOPE_LX_RESP* presp = lx_resp.add_rpt_lx_resp();
 presp->set_comm(command);
 presp->set_res(0);
 presp->set_reqid("0");

 LX_RESP_ENVELOPE_IN_COMMRESP* pc = lx_resp.mutable_commresp();

 AddTaskInfo1(pc->add_infolist());
 AddTaskInfo2(pc->add_infolist());
 AddTaskInfo3(pc->add_infolist());
 AddTaskInfo4(pc->add_infolist());
 AddTaskInfo5(pc->add_infolist());
 AddTaskInfo6(pc->add_infolist());
 AddTaskInfo7(pc->add_infolist());
 AddTaskInfo8(pc->add_infolist());
 AddTaskInfo9(pc->add_infolist());
 AddTaskInfo10(pc->add_infolist());
 AddTaskInfo11(pc->add_infolist());
 AddTaskInfo12(pc->add_infolist());
 AddTaskInfo13(pc->add_infolist());
 AddTaskInfo14(pc->add_infolist());
 AddTaskInfo15(pc->add_infolist());
 AddTaskInfo16(pc->add_infolist());
 AddTaskInfo17(pc->add_infolist());
 AddTaskInfo18(pc->add_infolist());
 AddTaskInfo19(pc->add_infolist());
 AddTaskInfo20(pc->add_infolist());
}

void Test::AddTaskInfo1(LX_RESP_ENVELOPE_IN_COMMRESP_INT_INFO* pt)
{
 pt->set_taskid("99");
 pt->set_resp_stat(0);
 pt->set_var1("localhost/aaaaaaaaaaaaaaa1");
 pt->set_var2("aaaaaaaaaaaaaaa1");
 pt->set_var5(99*10000);
 pt->set_var6(0);
}
void Test::AddTaskInfo2(LX_RESP_ENVELOPE_IN_COMMRESP_INT_INFO* pt)
{
 pt->set_taskid("98");
 pt->set_resp_stat(1);
 pt->set_var1("http://test/bbbbbbbbbbbbbb2");
 pt->set_var2("bbbbbbbbbbbbbb2");
 pt->set_var5(99*10000);
 pt->set_var6(1);
}

 /////////////////////////////// As said before. The First 14 INT_INFO data had been received and parsed correctly, problem occures on the 15th.

Original comment by qhlonl...@163.com on 26 Apr 2011 at 7:27

[GoogleCodeExporter]

That's showing the code used to populate the messages, but still none of the 
code doing the actual reading and writing.

Are you receiving the same number of bytes that you're sending?

If you try to parse the data you build (in the same process, with no 
possibility of data loss through transmission etc) on the C# side, what happens?

Original comment by jonsk...@google.com on 26 Apr 2011 at 8:51

[GoogleCodeExporter]

We had checked that we had received complete data from response side. And I had 
tried Response data built on C# side, It can be totally parsed by my C# request 
side, No such problem. But I still don't think that different programing 
language will make different. After all, If the response data less then 15 info 
dentry, We can parse it correctly. Is there any problem on our ".proto" file?

Regards!

Original comment by qhlonl...@163.com on 26 Apr 2011 at 12:50

[GoogleCodeExporter]

I wouldn't expect it to be *possible* to mess up anything in the proto file 
that would cause this.

So just to clarify, the data produced by the C# code can also be consumed by 
the C# code, but the data produced by the C# code fails in the C++ code? How 
have you validated that you've really received the data correctly on the C++ 
side, before parsing?

Can you attach a sample file that demonstrates the problem?

Could you also clarify which side you're calling request and which is response, 
and where the C# and C++ are? Sometimes you're talking about C# server code, 
and sometimes C++ server code...

Original comment by jonsk...@google.com on 26 Apr 2011 at 12:54

[GoogleCodeExporter]

Oh, I forgot to describe a detail. We using HTTP protocol to warp the ProtoBuf 
data, So we read the response data from HttpResponse.Stream. We thought that 
since we had pass the response data as Http Body (where Http Header set 
Content_type: application/octet-stream), there will be no data change with 
transport process.
When I had received the response data as byte[] DATA, I simply use  
global::LX_RESP_ENVELOPE.ParseFrom(DATA) To get the data structure，And use 
it. When there is more then 14 submessages, the ParseFrom will raise exception 
info as reported before.

Original comment by qhlonl...@163.com on 26 Apr 2011 at 1:02

[GoogleCodeExporter]

Our request side using C# code, while response side use C++.

Original comment by qhlonl...@163.com on 26 Apr 2011 at 1:05

[GoogleCodeExporter]

So how are you reading the data from HttpResponse.Stream, and how are you 
checking that you still have the right data? You claim that "there will be no 
data change" - but have you verified that? Have you checked that you've 
*actually* read the right data?

It's pretty easy to read data incorrectly from an HTTP stream - and not 
impossible to *write* it badly too... but we still haven't seen that bit of 
code.

I suggest that for test purposes, you create two small *console* applications - 
one in C# and one in C++. (It's still unclear to me what's writing and what's 
reading this data.) Basically, write the data to a file in one app, and read it 
from the file in the other app. To read/write the file correctly in C#, use 
File.ReadAllBytes or File.WriteAllBytes.

Original comment by jonsk...@google.com on 26 Apr 2011 at 1:07

[GoogleCodeExporter]

Sorry for my carelessness on Comment4, The C# code pasted are the request side, 
or Client side.

Original comment by qhlonl...@163.com on 26 Apr 2011 at 1:10

[GoogleCodeExporter]

Okay, so you're *creating* the data in C++, writing it to an HTTP response, and 
then *reading* it in C# from an HttpResponse.Stream. Please show the code 
you're using to read the response body before parsing it.

Original comment by jonsk...@google.com on 26 Apr 2011 at 1:15

[GoogleCodeExporter]

Thanks for your suggestion. I had justed verified that I had received the same 
length of bytes with that send from C++ Server. (C++ side is the server side, 
It gives resposne, my C# side is a client side to send request.)

Original comment by qhlonl...@163.com on 26 Apr 2011 at 1:18

[GoogleCodeExporter]

The HttpResponse Is a Simple warp of Stream, HttpResponse.Stream is just a .NET 
Stream type.
I now give the request side code to get response data.
""""
length = Int32.Parse(HttpHeader["Content-Length"]); // It get the same length 
of data as C++ side send to me. verified.
byte[] recvdata = new byte[length];
resposne.Stream.Read(recvdata, 0, length );
""""
So here "recvdata" is my received data from C++ side.

Regards！

Original comment by qhlonl...@163.com on 26 Apr 2011 at 1:31

[GoogleCodeExporter]

Don't just check that you've got the same number of bytes - check that the data 
*within the byte array* is the same. For example, here's some broken code:

    byte[] data = new byte[response.ContentLength];
    // Ignores the return value
    responseStream.Read(data, 0, data.Length);

*Please* could you just post the code which reads the data. I've asked for this 
right from the very first comment, and there's still no sign of it, which is 
making it impossible to help you...

Original comment by jonsk...@google.com on 26 Apr 2011 at 1:31

[GoogleCodeExporter]

Right, I thought so. Your reading code is broken. You're ignoring the return 
value of Stream.Read. You should loop:

   length = Int32.Parse(HttpHeader["Content-Length"]);
   byte[] recvdata = new byte[length];
   int index = 0;
   while (index < recvdata.Length)
   {
       int bytesRead = resposne.Stream.Read(recvdata, index, recvdata.Length - index);
       if (bytesRead <= 0) // End of stream
       {
           throw new IOException("Unable to read all data");
       }
       index += bytesRead;
   }

Original comment by jonsk...@google.com on 26 Apr 2011 at 1:36

[GoogleCodeExporter]

Oh，Thank you very much. I will have a try.

Original comment by qhlonl...@163.com on 26 Apr 2011 at 1:42

[GoogleCodeExporter]

Yes, Now It goes well. Sorry for interupte you so much with my low-level 
program code bug. and Thank you again for your kindly help! 

Regards!

Original comment by qhlonl...@163.com on 26 Apr 2011 at 2:06

[GoogleCodeExporter]

Closing as invalid.

Original comment by jonathan.skeet on 26 Apr 2011 at 2:11

• Changed state: Invalid