SerializeAsString - server, ParseFromString - client. "Decode error" Exception occurred, but the data looks correct.

[itzikban]

Hi, Our application workflow uses protobuf for cross-language IPC. We have a C++ application publishing data over Linux shared memory to various clients on the same host. Data is published to shared memory using the protobuf API of "SerializeAsString", and the client's side does "ParseFromString". Some of the clients are written in Python, while others are written in C++. Even though the data we get after parsing appears to be fine, in C++ the "ParseFromString" method always returns false. In the beginning we used protobuf v3.15.5 on the Python clients, we got "RuntimeWarning: Unexpected end-group tag: Not all data was converted", from ParseFromString() call.

After upgrading server and client side protobuf to 21.12, we start getting Decode error exception. Again, the strange thing is that all the data looks fine despite the exceptions. Any suggestions?

Language: C++//Python

operating system :

• Server - Docker image Ubuntu 20.04.5 run on aarch64.

• Client - Docker image Ubuntu 20.04.5 run on x86.

  Runtime / compiler are you using - Python 3.10, Gcc/G++ - 9.

What did you do? Steps to reproduce the behavior:

Part of my Proto:

message Frame
{
  bytes depth           = 1;
  bytes rgb             = 2;
  uint64 SampleTime     = 3;
  uint64 SentTime       = 4;
  uint64 AlignTime      = 5;
}
message CameraData
{
  Frame                 frame           = 1;
  uint32                fps             = 2;
}

Serialize with: SerializeAsString() Parse with : ParseFromString()

Thanks a lot for your Help !

[shaod2]

Can you give an example of the failing data? And make sure that it is Protobuf binary wire format, not protobuf text format.

[itzikban]

Hi @shaod2 thanks for your respond here is the data data.zip Or read it with python

data1:
b'\rQ\xc3\x8e\xb6\x15\xd6\xae\xc7\xb8\x1d\x01m-\xb7%\xe6\xb2@?-\x12\xef\x11?5\xec\xf00>=\xdb\x85\x8f>E;\xd3j\xb9M\xa49\xa4\xbbUj\xa1\x0e\xba]\xfcL1\xbaeI\x03\x04\xbbmD\x98\xf7\xbau\xa3\x10\xc2\xbb}*\xb8\x07\xbe\x85\x01\x0f\xbfk\xbc\x8d\x01\x8f\xcc\xe7>\x95\x01\x92I0\xbf\x9d\x01\xee\xa2\xef\xbf\xa0\x01\x02\xad\x01\xff\xff\x7f\x7f\xb5\x01\xff\xff\x7f\x7f\xc0\x01\xd5\xc4\x9b\xaf\x91\xf4\xfc\x02\xc8\x01\xdb\xf2\x9b\xaf\x91\xf4\xfc\x02\xd2\x01\x02\x08\x01'

data2:
b'\rt"\x1d5\x15\xa0\xbc\x04\xb9\x1d\tF\x18\xb7%\x1e\xb3@?-\x8f\xee\x11?5\x93\xf10>=\x9b\x86\x8f>E\xa6\xe7\xad9M\xf1\xbf\xc9\xbbU\x8f\xb9J\xb9]7\xf5\x10\xbbe\xabv(\xbbm\xc1\x1bl\xb9u\x19\x1a\xed<}\xacH&\xbe\x85\x01\xd2/;<\x8d\x01\xc1\xb6_\xbf\x95\x01\xac\xd0\xa4>\x9d\x01&\xd1\xfd?\xa0\x01\x02\xad\x01\xff\xff\x7f\x7f\xb5\x01\xff\xff\x7f\x7f\xc0\x01\xe9\xeb\x9b\xaf\x91\xf4\xfc\x02\xc8\x01\x98\xbd\x9c\xaf\x91\xf4\xfc\x02\xd2\x01\x02\x08\x01'

data3:
b'\r\x82\xd1[6\x15-eM\xb9\x1d\n\xc6C\xb7%i\xb2@?-\xa8\xee\x11?5\x9f\xf20>=\xa7\x89\x8f>E\xeb\xbf\xdc9M\xb2:\xfd\xbbU\x9bsN\xb9]\x96\xcb\x92\xbbeg\xe6\xfc\xbaml\xf04\xbau\xc0\x10\xa5<}I\xba#\xbe\x85\x01\x1c\xee\xcc;\x8d\x01&4\x03\xbf\x95\x01a\x11\xc1?\x9d\x01\xd89\xa8>\xa0\x01\x02\xad\x01\xff\xff\x7f\x7f\xb5\x01\xff\xff\x7f\x7f\xc0\x01\x91\xba\x9c\xaf\x91\xf4\xfc\x02\xc8\x01\xa2\xff\x9c\xaf\x91\xf4\xfc\x02\xd2\x01\x02\x08\x01'

Thanks for your help!

Can you give an example of the failing data? And make sure that it is Protobuf binary wire format, not protobuf text format.

[github-actions[bot]]

We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please add a comment.

This issue is labeled inactive because the last activity was over 90 days ago.

[github-actions[bot]]

We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please reopen it.

This issue was closed and archived because there has been no new activity in the 14 days since the inactive label was added.