Unexpected end of buffer in varint

[piperoc]

I created a simple testing solution.

My proto file looks like this:

syntax = "proto2";

message SimpleMessage {
  required int32 msg_id = 1;
  optional string msg_string = 2;
}

message ComplexMessage {
  required int32 complex_msg_id = 1;
  repeated SimpleMessage messages = 2;
}

The codegen processes it without any problem and eventually translated the repeated SimpleMessage into std::vector<SimpleMessage> as you would expect.

My code sends and receives through a TCP socket.

I can encode/decode SimpleMessage object without any issue. But in trying to decode a ComplexMessage I run into the error: Unexpected end of buffer in varint which seems to be related to an EOF of sorts.

I will try to simplify my test to take out the socket code so I can share a simpler test. But in the meanwhile I wonder if there is any size limitation that my vector might exceed.

Thanks again.

[piperoc]

I simplified the code to show my issue.

Here's the codegen cpp file from the proto above:

messages.pb.cpp

```cpp // Generated by a ProtoBuf compiler. DO NOT EDIT! // Source: messages.pbs #include #include #include struct SimpleMessage { int32_t msg_id = 0; std::string msg_string; bool has_msg_id = false; bool has_msg_string = false; #ifdef EASYPB_SimpleMessage_EXTRA_FIELDS EASYPB_SimpleMessage_EXTRA_FIELDS #endif void encode(easypb::Encoder &pb) const; void decode(easypb::Decoder pb); }; void SimpleMessage::encode(easypb::Encoder &pb) const { pb.put_int32(1, msg_id); pb.put_string(2, msg_string); #ifdef EASYPB_SimpleMessage_EXTRA_ENCODING EASYPB_SimpleMessage_EXTRA_ENCODING #endif } void SimpleMessage::decode(easypb::Decoder pb) { while(pb.get_next_field()) { switch(pb.field_num) { case 1: pb.get_int32(&msg_id, &has_msg_id); break; case 2: pb.get_string(&msg_string, &has_msg_string); break; #ifdef EASYPB_SimpleMessage_EXTRA_DECODING EASYPB_SimpleMessage_EXTRA_DECODING #endif default: pb.skip_field(); } } #ifdef EASYPB_SimpleMessage_EXTRA_POST_DECODING EASYPB_SimpleMessage_EXTRA_POST_DECODING #endif if(! has_msg_id) { throw easypb::missing_required_field("Decoded protobuf has no required field SimpleMessage.msg_id"); } } struct ComplexMessage { int32_t complex_msg_id = 0; std::vector messages; bool has_complex_msg_id = false; #ifdef EASYPB_ComplexMessage_EXTRA_FIELDS EASYPB_ComplexMessage_EXTRA_FIELDS #endif void encode(easypb::Encoder &pb) const; void decode(easypb::Decoder pb); }; void ComplexMessage::encode(easypb::Encoder &pb) const { pb.put_int32(1, complex_msg_id); pb.put_repeated_message(2, messages); #ifdef EASYPB_ComplexMessage_EXTRA_ENCODING EASYPB_ComplexMessage_EXTRA_ENCODING #endif } void ComplexMessage::decode(easypb::Decoder pb) { while(pb.get_next_field()) { switch(pb.field_num) { case 1: pb.get_int32(&complex_msg_id, &has_complex_msg_id); break; case 2: pb.get_repeated_message(&messages); break; #ifdef EASYPB_ComplexMessage_EXTRA_DECODING EASYPB_ComplexMessage_EXTRA_DECODING #endif default: pb.skip_field(); } } #ifdef EASYPB_ComplexMessage_EXTRA_POST_DECODING EASYPB_ComplexMessage_EXTRA_POST_DECODING #endif if(! has_complex_msg_id) { throw easypb::missing_required_field("Decoded protobuf has no required field ComplexMessage.complex_msg_id"); } } ```

And this is the main program to test encode/decode and get the error:

main.cpp

```cpp #include #include #include #include "messages.pb.cpp" SimpleMessage makeSimpleMessage(int32_t mid, const std::string& ms) { SimpleMessage message; message.msg_id = mid; message.msg_string = ms; return message; } ComplexMessage makeComplexMessage(int32_t cid, std::vector simple_vector) { ComplexMessage message; message.complex_msg_id = cid; message.messages = simple_vector; return message; } int main() { std::vector simple_vector = {makeSimpleMessage(1, "Hello"), makeSimpleMessage(2, "World")}; ComplexMessage complex_message = makeComplexMessage(1, simple_vector); auto enc = easypb::encode(complex_message); const char * encoded = enc.c_str(); std::cout << "Encoded: " << encoded << std::endl; auto dec = easypb::decode(encoded); std::cout << "Decoded: " << dec.complex_msg_id << std::endl; for(auto& msg : dec.messages) { std::cout << "--- Message: " << msg.msg_id << " " << msg.msg_string << std::endl; } std::cout << std::endl; return 0; } ```

I look at the generated code and it seems to have all the necessary code to deal with the std::vector<SimpleMessage> . Unlike the tutorial example, I would not know how to add those

EASYPB_MainMessage_EXTRA_FIELDS   
EASYPB_MainMessage_EXTRA_ENCODING 
EASYPB_MainMessage_EXTRA_DECODING 

since the generated code already has the expected lines...

Thanks for clarifying.

[Bulat-Ziganshin]

That's easy - you should pass std::string to the decoder, since "char*" type doesn't carry the string size information. Change the decoding call to

easypb::decode(enc);

All those EXTRA macros can be used to manually handle complex cases where Codegen can't auto-generate the code. It's not the case here. Of course, I will document these macros, today Codegen docs are almost non-existent.