Deserialization ignores a nonempty field

[GoogleCodeExporter]

What steps will reproduce the problem?

1. Deserialize a ProductDefinition message from base-64 encoded byte array  
CgNFVVIQgIDBraOr+OgIGAEhexSuR+F6hD8oADABOABAAUgAUAFYAGEAAAAAAECPQGoTUFJPRC5GLkZH
QkwuREVDMjAxNHIKRkdCTCBERUMxNHgCgAEBigEzCAEQABiAgJrakJL46AgggKjrjJmFwagIKgdERUMy
MDE0MP///////////wE4DEDeD0oAmgEOCgxERTAwMDk2NTI2NDSgAbEIqgEMUlhaMTRfQ29tZHR5 

2. Expected value has a nonempty QTGId field 
 { Currency: "EUR" DeletionDate: 635536800000000000 Environment: ProductionEnvironment IncrementPrice: 0.01 IsBasket: false IsDerivative: true IsOption: false IsOutright: true IsStrategy: false IsValid: true IsWildcard: false PointValue: 1000 Product: "PROD.F.FGBL.DEC2014" ProductName: "FGBL DEC14" ProductType: FutureProductType ProductSource: EnumProductSourceSnapshot Derivative { CashSettled: true ExpirationSettlementDate: 0 LastTradingDate: 635535936000000000 LastTradingTime: 599264802000000000 Series: "DEC2014" SeriesDayOfMonth: -1 SeriesMonth: 12 SeriesYear: 2014 UnderlyingProduct: "" } Outright { ISIN: "DE0009652644" } QTGId: 1073 QTGName: "RXZ14_Comdty" }

3. Actual value has an empty QTGId field

Tested with versions 2.3.0.277 and 2.4.1.521

Original issue reported on code.google.com by jko...@liquidcapital.com on 22 Oct 2014 at 10:12

Attachments:

• ProductDefinitions.proto

[GoogleCodeExporter]

It would be helpful if you could reproduce this with a simpler message - 
ideally showing what *created* that data first (to prove that it started with a 
QTGId value).

Original comment by jonathan.skeet on 22 Oct 2014 at 10:14

[GoogleCodeExporter]

The message was generated and logged in production environment. The same 
message object was converted into a log item string like so (C#) :

- var logItem = message.ToString().Replace("\n", " ") // the human readable 
format

-   var buffer = message.ToByteArray(); 
    var logItem = Convert.ToBase64String(buffer); // the base-64 format

However, deserialization from the base-64 format yielded a message with an  
empty QTGId 

Original comment by jko...@liquidcapital.com on 22 Oct 2014 at 10:23

[GoogleCodeExporter]

The value 1073 is stored in field 20 in the data - which is meant to be a 
BondProduct. Likewise there's a string value ("RXZ14_Comdty") in field 21, 
which is meant to be QTGId.

If you change the proto so that QTGId is field 20, and QTGName is field 21, 
then it works fine.

My strong suspicion is that someone changed your proto file after you'd already 
serialized the data, and renumbered the fields. You should never do that, 
because it causes precisely this kind of problem.

Original comment by jonathan.skeet on 22 Oct 2014 at 10:48

[GoogleCodeExporter]

Thanks, that would explain the problem.

Original comment by jko...@liquidcapital.com on 22 Oct 2014 at 10:51

[GoogleCodeExporter]

Fyi, there was indeed a change in the ProductDefinitions.proto file
---    required int32 QTGId = 20;
---    required string QTGName = 21;

+++    required int32 QTGId = 21;
+++    required string QTGName = 22;

Original comment by jko...@liquidcapital.com on 22 Oct 2014 at 10:57

[GoogleCodeExporter]

In that case, I'm afraid the answer is "don't do that". The fields are keyed by 
number, so if you change the numbers after storing data, you *will* experience 
problems. The code is working as intended.

If you haven't stored anything with the *new* version of the proto, you can 
just go back to the old version and all should be well. If you've got a mixture 
of data stored with the new version and data stored with the old version, 
you've got a bit of a data cleaning exercise to go through.

Original comment by jonathan.skeet on 22 Oct 2014 at 11:43

• Changed state: Invalid