view toObject data mapping missing values

[imamark]

Hi @TimelordUK, Great work on this library thank you very much.

I have an issue that I wondered if you could help me with?

We're requesting multiple MDFullGrp > MDEntry[] values in a Market Data Request (Bid,Offer and Mid) and the response is not getting mapped in the toObject method correctly.

Response ASCII

[0] 8 (BeginString) = FIX.4.4, 
[1] 9 (BodyLength) = 224
[2] 35 (MsgType) = W[MarketDataSnapshotFullRefresh], 
[3] 34 (MsgSeqNum) = 8
[4] 49 (SenderCompID) = TEST, 
[5] 56 (TargetCompID) = TEST
[6] 52 (SendingTime) = 20220621-17:16:16.414, 
[7] 262 (MDReqID) = #GBPUSD#0#
[8] 55 (Symbol) = GBPUSD, 
[9] 268 (NoMDEntries) = 3
[10] 269 (MDEntryType) = 0[Bid], 
[11] 270 (MDEntryPx) = 1.22759
[12] 271 (MDEntrySize) = 1, 
[13] 63 (SettlType) = 0[Regular]
[14] 272 (MDEntryDate) = 20220623, 
[15] 768 (NoTrdRegTimestamps) = 0
[16] 269 (MDEntryType) = 1[Offer], 
[17] 270 (MDEntryPx) = 1.22759
[18] 271 (MDEntrySize) = 1, 
[19] 63 (SettlType) = 0[Regular]
[20] 272 (MDEntryDate) = 20220623, 
[21] 768 (NoTrdRegTimestamps) = 0
[22] 269 (MDEntryType) = H[MidPrice], 
[23] 270 (MDEntryPx) = 1.22759
[24] 63 (SettlType) = 0[Regular], 
[25] 272 (MDEntryDate) = 20220623
[26] 768 (NoTrdRegTimestamps) = 0, 
[27] 10 (CheckSum) = 066

Mapped Object

{
  StandardHeader: {
    BeginString: 'FIX.4.4',
    BodyLength: 224,
    MsgType: 'W',
    SenderCompID: 'TEST',
    TargetCompID: 'TEST',
    MsgSeqNum: 8,
    SendingTime: 2022-06-21T17:16:16.414Z
  },
  MDReqID: '#GBPUSD#0#',
  Instrument: { Symbol: 'GBPUSD' },
  MDFullGrp: [
    {
      MDEntryType: '0',
      MDEntryPx: 1.22759,
      MDEntrySize: 1,
      MDEntryDate: 2022-06-23T00:00:00.000Z,
      SettlType: '0'
    }
  ],
  StandardTrailer: { CheckSum: '066' }
}

Here i would expect the MDFullGrp array to contain the 3 values.

Is there anything I'm missing or will I need to fix it in the code and create a PR?

Thanks in advance

[TimelordUK]

hello,

I have not had much time to look at this but

I made changes here to look at https://github.com/TimelordUK/jspf-md-demo

note the changes required in FIX44-MD.xml e.g.

 <component name='MDFullGrp'>
   <group name='NoMDEntries' required='Y'>
    <field name='MDEntryType' required='Y' />
    <field name='MDEntryPx' required='N' />
    <field name='Currency' required='N' />
    <field name='MDEntrySize' required='N' />
    <field name='SettlType' required='N' />
    <component name='TrdRegTimestamps' required='N' />
    <field name='MDEntryDate' required='N' />
    <field name='MDEntryTime' required='N' />
    <field name='TickDirection' required='N' />
    <field name='MDMkt' required='N' />
    <field name='TradingSessionID' required='N' />
    <field name='TradingSessionSubID' required='N' />
    <field name='QuoteCondition' required='N' />
    <field name='TradeCondition' required='N' />
    <field name='MDEntryOriginator' required='N' />
    <field name='LocationID' required='N' />
    <field name='DeskID' required='N' />
    <field name='OpenCloseSettlFlag' required='N' />
    <field name='TimeInForce' required='N' />
    <field name='ExpireDate' required='N' />
    <field name='ExpireTime' required='N' />
    <field name='MinQty' required='N' />
    <field name='ExecInst' required='N' />
    <field name='SellerDays' required='N' />
    <field name='OrderID' required='N' />
    <field name='QuoteEntryID' required='N' />
    <field name='MDEntryBuyer' required='N' />
    <field name='MDEntrySeller' required='N' />
    <field name='NumberOfOrders' required='N' />
    <field name='MDEntryPositionNo' required='N' />
    <field name='Scope' required='N' />
    <field name='PriceDelta' required='N' />
    <field name='Text' required='N' />
    <field name='EncodedTextLen' required='N' />
    <field name='EncodedText' required='N' />
   </group>
  </component>

we build a response on server

 public static FullSnapshot (symbol: string): ILooseObject {
    return {
      MDReqID: `#${symbol}#0#`, // #GBPUSD#0#
      Instrument: {
        SecurityID: symbol
      },
      MDFullGrp: {
        NoMDEntries: [
          {
            MDEntryType: MDEntryType.Bid,
            MDEntryPx: 1.22759,
            MDEntrySize: 1,
            SettlType: SettlType.Regular,
            MDEntryDate: new Date('20220623'),
            TrdRegTimestamps: {
              ITrdRegTimestampsNoTrdRegTimestamps: []
            } as ITrdRegTimestamps
          } as IMDFullGrpNoMDEntries,
          {
            MDEntryType: MDEntryType.Offer,
            MDEntryPx: 1.22759,
            MDEntrySize: 1,
            SettlType: SettlType.Regular,
            MDEntryDate: new Date('20220623'),
            TrdRegTimestamps: {
              ITrdRegTimestampsNoTrdRegTimestamps: []
            } as ITrdRegTimestamps
          } as IMDFullGrpNoMDEntries,
          {
            MDEntryType: MDEntryType.MidPrice,
            MDEntryPx: 1.22759,
            MDEntrySize: 1,
            SettlType: SettlType.Regular,
            MDEntryDate: new Date('20220623'),
            TrdRegTimestamps: {
              ITrdRegTimestampsNoTrdRegTimestamps: []
            } as ITrdRegTimestamps
          } as IMDFullGrpNoMDEntries
        ]
      } as IMDFullGrp
    } as IMarketDataSnapshotFullRefresh
  }

the response from server turned into object

{    
"StandardHeader": {
        "BeginString": "FIX4.4",
        "BodyLength": 218,
        "MsgType": "W",
        "SenderCompID": "accept-comp",
        "TargetCompID": "init-comp",
        "MsgSeqNum": 2,
        "TargetSubID": "fix",
        "SendingTime": "2022-06-21T19:16:44.298Z"
    },
    "MDReqID": "#GBPUSD#0#",
    "Instrument": {
        "SecurityID": "GBPUSD"
    },
    "MDFullGrp": {
        "NoMDEntries": [
            {
                "MDEntryType": "0",
                "MDEntryPx": 1.22759,
                "MDEntrySize": 1,
                "SettlType": "0"
            },
            {
                "MDEntryType": "1",
                "MDEntryPx": 1.22759,
                "MDEntrySize": 1,
                "SettlType": "0"
            },
            {
                "MDEntryType": "H",
                "MDEntryPx": 1.22759,
                "MDEntrySize": 1,
                "SettlType": "0"
            }
        ]
    },
    "StandardTrailer": {
        "CheckSum": "125"
    }
}

[imamark]

Thanks @TimelordUK that was really helpful to resolve the issue. Awesome work!

[dondopong]

Hi @TimelordUK @imamark I'm facing this issue in v3.0.0

This is what the server sends (SecurityList 35=y):

8=FIXT.1.1|9=1247|35=y|34=36|49=MATCHING1|52=20230301-00:26:26.316|56=drojas|320=01|322=107074672621291|393=223|559=4|560=0|893=Y|1300=DDA|1301=ROFX|146=3|55=SOJ.ROS/ENE23|461=FXXXSX|200=202301|541=20230228|228=1.000000|5023=1.000000|5514=1|7117=0|231=100.000000|207=ROFX|107=SOJ.ROS/ENE23|969=0.100000|711=1|311=Soja Rosario|15=USD|9996=0|1205=1|1206=0|1208=0.100000|1234=1|1093=3|1231=5.000000|1148=349.4|1149=409.4|562=1.000000|1140=20.000000|561=1.000000|1309=1|336=10|1237=3|40=K|40=2|40=4|1239=4|59=0|59=6|59=4|59=3|55=TRI.ROS/ENE23|461=FXXXSX|200=202301|541=20230124|228=1.000000|5023=1.000000|5514=1|7117=0|231=100.000000|207=ROFX|107=TRI.ROS/ENE23|969=0.100000|711=1|311=Trigo Rosario|15=USD|9996=36400|1205=1|1206=0|1208=0.100000|1234=1|1093=3|1231=5.000000|1148=258.0|1149=312.0|562=1.000000|1140=20.000000|561=1.000000|1309=1|336=10|1237=3|40=K|40=2|40=4|1239=4|59=0|59=4|59=6|59=3|55=MAI.ROS/ENE23|461=FXXXSX|200=202301|541=20230124|228=1.000000|5023=1.000000|5514=1|7117=0|231=100.000000|207=ROFX|107=MAI.ROS/ENE23|969=0.100000|711=1|311=Maíz Rosario|15=USD|9996=40300|1205=1|1206=0|1208=0.100000|1234=1|1093=3|1231=5.000000|1148=212.0|1149=246.0|562=1.000000|1140=20.000000|561=1.000000|1309=1|336=10|1237=3|40=K|40=2|40=4|1239=4|59=0|59=6|59=4|59=3|10=026|

Client maps to:

{
    "StandardHeader": {
        "BeginString": "FIXT.1.1",
        "BodyLength": 1247,
        "MsgType": "y",
        "SenderCompID": "MATCHING1",
        "TargetCompID": "drojas",
        "MsgSeqNum": 36,
        "SendingTime": "2023-03-01T00:26:26.316Z"
    },
    "SecurityReqID": "01",
    "SecurityResponseID": "107074672621291",
    "SecurityRequestResult": 0,
    "SecurityListRequestType": 4,
    "TotNoRelatedSym": 223,
    "LastFragment": true,
    "SecListGrp": {
        "NoRelatedSym": [
            {
                "Instrument": {
                    "Symbol": "SOJ.ROS/ENE23",
                    "CFICode": "FXXXSX",
                    "MaturityMonthYear": "202301",
                    "MaturityDate": "2023-02-28T00:00:00.000Z",
                    "Factor": 1,
                    "TickSize": 1,
                    "InstrumentPricePrecision": 1,
                    "InstrumentSizePrecision": 0,
                    "ContractMultiplier": 100,
                    "SecurityExchange": "ROFX",
                    "SecurityDesc": "SOJ.ROS/ENE23",
                    "MinPriceIncrement": 0.1
                },
                "UndInstrmtGrp": {
                    "NoUnderlyings": [
                        {
                            "UnderlyingInstrument": {
                                "UnderlyingSymbol": "Soja Rosario"
                            }
                        }
                    ]
                },
                "Currency": "USD",
                "ContractPositionNumber": 0
            }
        ]
    },
    "MarketID": "ROFX",
    "MarketSegmentID": "DDA",
    "StandardTrailer": {
        "CheckSum": "026"
    }
}

This is my dictionary definition:

  <message name="SecurityList" msgtype="y" msgcat="app">
            <field name="SecurityReqID" required="N"/>
            <field name="SecurityResponseID" required="N"/>
            <field name="SecurityRequestResult" required="N"/>
            <field name="SecurityListRequestType" required="N" />
            <field name="TotNoRelatedSym" required="N"/>
            <field name="LastFragment" required="N"/>
            <component name="SecListGrp" required="N"/>
            <field name="SecurityReportID" required="N"/>
            <field name="ClearingBusinessDate" required="N"/>
            <field name="MarketID" required="N" />
            <field name="MarketSegmentID" required="N" />
            <field name="SecurityListID" required="N" />
            <field name="SecurityListRefID" required="N" />
            <field name="SecurityListDesc" required="N" />
            <field name="EncodedSecurityListDescLen" required="N" />
            <field name="EncodedSecurityListDesc" required="N" />
            <field name="SecurityListType" required="N" />
            <field name="SecurityListTypeSource" required="N" />
            <field name="TransactTime" required="N" />
        </message>

        <component name="SecListGrp">
            <group name="NoRelatedSym" required="N">
                <component name="Instrument"/>
                <component name="InstrumentExtension"/>
                <component name="FinancingDetails"/>
                <component name="UndInstrmtGrp"/>
                <field name="Currency" required="N"/>
                <field name="ContractPositionNumber" required="N"/>
                <component name="Stipulations"/>
                <component name="InstrmtLegSecListGrp"/>
                <component name="SpreadOrBenchmarkCurveData"/>
                <component name="YieldData"/>
                <field name="Text" required="N"/>
                <field name="EncodedTextLen" required="N"/>
                <field name="EncodedText" required="N"/>
                <component name="SecurityTradingRules" required="N" />
                <component name="StrikeRules" required="N" />
                <field name="RelSymTransactTime" required="N" />
            </group>
        </component>

Same thing. Group [NoRelatedSym] only maps its first value. How can I solve this? Thanks in advance. BTW, I'm not having any trouble with MarketData repetitive groups, just with these ones.

[TimelordUK]

I will look more into it later this is usually related to missing fields in the data dictionary in fact if you look back there has been a previous issue raised virtually identical to this one. Basically your vendor has additional fields they normally provide some sort of data dictionary api document containing what they are. Essentially when parsing a group the parsing stops when it hits a field it does not know about in group. Look at the dictionary and add the missing fields.   It’s much easier if you are using a quickfix related dict on client. Is this ICE if so you can ask for the latest definition for say for trade capture and add the fields onto underlying security. I will look further later if you send the vendor api definition I can help Sent from my iPadOn 1 Mar 2023, at 00:39, dondopong @.***> wrote: Hi @TimelordUK @imamark I'm facing this issue in v3.0.0 This is what the server sends (SecurityList 35=y): 8=FIXT.1.1|9=1247|35=y|34=36|49=MATCHING1|52=20230301-00:26:26.316|56=drojas|320=01|322=107074672621291|393=223|559=4|560=0|893=Y|1300=DDA|1301=ROFX|146=3|55=SOJ.ROS/ENE23|461=FXXXSX|200=202301|541=20230228|228=1.000000|5023=1.000000|5514=1|7117=0|231=100.000000|207=ROFX|107=SOJ.ROS/ENE23|969=0.100000|711=1|311=Soja Rosario|15=USD|9996=0|1205=1|1206=0|1208=0.100000|1234=1|1093=3|1231=5.000000|1148=349.4|1149=409.4|562=1.000000|1140=20.000000|561=1.000000|1309=1|336=10|1237=3|40=K|40=2|40=4|1239=4|59=0|59=6|59=4|59=3|55=TRI.ROS/ENE23|461=FXXXSX|200=202301|541=20230124|228=1.000000|5023=1.000000|5514=1|7117=0|231=100.000000|207=ROFX|107=TRI.ROS/ENE23|969=0.100000|711=1|311=Trigo Rosario|15=USD|9996=36400|1205=1|1206=0|1208=0.100000|1234=1|1093=3|1231=5.000000|1148=258.0|1149=312.0|562=1.000000|1140=20.000000|561=1.000000|1309=1|336=10|1237=3|40=K|40=2|40=4|1239=4|59=0|59=4|59=6|59=3|55=MAI.ROS/ENE23|461=FXXXSX|200=202301|541=20230124|228=1.000000|5023=1.000000|5514=1|7117=0|231=100.000000|207=ROFX|107=MAI.ROS/ENE23|969=0.100000|711=1|311=Maíz Rosario|15=USD|9996=40300|1205=1|1206=0|1208=0.100000|1234=1|1093=3|1231=5.000000|1148=212.0|1149=246.0|562=1.000000|1140=20.000000|561=1.000000|1309=1|336=10|1237=3|40=K|40=2|40=4|1239=4|59=0|59=6|59=4|59=3|10=026| Client maps to: { "StandardHeader": { "BeginString": "FIXT.1.1", "BodyLength": 1247, "MsgType": "y", "SenderCompID": "MATCHING1", "TargetCompID": "drojas", "MsgSeqNum": 36, "SendingTime": "2023-03-01T00:26:26.316Z" }, "SecurityReqID": "01", "SecurityResponseID": "107074672621291", "SecurityRequestResult": 0, "SecurityListRequestType": 4, "TotNoRelatedSym": 223, "LastFragment": true, "SecListGrp": { "NoRelatedSym": [ { "Instrument": { "Symbol": "SOJ.ROS/ENE23", "CFICode": "FXXXSX", "MaturityMonthYear": "202301", "MaturityDate": "2023-02-28T00:00:00.000Z", "Factor": 1, "TickSize": 1, "InstrumentPricePrecision": 1, "InstrumentSizePrecision": 0, "ContractMultiplier": 100, "SecurityExchange": "ROFX", "SecurityDesc": "SOJ.ROS/ENE23", "MinPriceIncrement": 0.1 }, "UndInstrmtGrp": { "NoUnderlyings": [ { "UnderlyingInstrument": { "UnderlyingSymbol": "Soja Rosario" } } ] }, "Currency": "USD", "ContractPositionNumber": 0 } ] }, "MarketID": "ROFX", "MarketSegmentID": "DDA", "StandardTrailer": { "CheckSum": "026" } }

Same thing. Group [NoRelatedSym] only maps its first value. How can I solve this? Thanks in advance.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

[dondopong]

@TimelordUK Thanks a lot for your quick reply! I checked the difference between incoming messages and generated interfaces, and I found out that some interfaces have been generated incorrectly using the generator. Could you please check yourself anytime if I'm doing things right?

This is the xml file that I'm using as dictionary when running the generator: https://github.com/dondopong/randomStuff/blob/main/fix50modified.xml

I'm having trouble in particular with the component BaseTradingRules:

        <component name="BaseTradingRules">
            <component name="TickRules" required="N" />
            <component name="LotTypeRules" required="N" />
            <component name="PriceLimits" required="N" />
            <field name="ExpirationCycle" required="N" />
            <field name="MinTradeVol" required="N" />
            <field name="MaxTradeVol" required="N" />
            <field name="MaxPriceVariation" required="N" />
            <field name="ImpliedMarketIndicator" required="N" />
            <field name="TradingCurrency" required="N" />
            <field name="RoundLot" required="N" />
            <field name="MultilegModel" required="N" />
            <field name="MultilegPriceMethod" required="N" />
            <field name="PriceType" required="N" />
        </component>

And this is what's generated from that:

export interface IBaseTradingRules {
  ExpirationCycle?: number// [1] 827 (Int)
  MinTradeVol?: number// [2] 562 (Float)
  MaxTradeVol?: number// [3] 1140 (Float)
  MaxPriceVariation?: number// [4] 1143 (Float)
  ImpliedMarketIndicator?: number// [5] 1144 (Int)
  TradingCurrency?: string// [6] 1245 (String)
  RoundLot?: number// [7] 561 (Float)
  MultilegModel?: number// [8] 1377 (Int)
  MultilegPriceMethod?: number// [9] 1378 (Int)
  PriceType?: number// [10] 423 (Int)
}

Seems that those sub-components are missing, despite the fact that they are defined in the xml. I would really appreciate your help! Thanks!

[TimelordUK]

does this work - there may be an issue with forward references

i just moved your components before the main component

[dondopong]

Thank you, it worked like a charm! I'll open an issue to review this later.