Return integer data type

[jlroo]

I was wondering if its possible that during the decoding process from FIX to json, instate of returning strings " " as output ei. ( "MDEntryPx" : "167250" ) it returns an integer data type ei. ( "MDEntryPx" : 167250 ) currently I am using a mongo script for this but would love if this can be achieve during the decoding process.

[salsferrazza]

Hi jlroo,

I almost hadn't realized that it fix2json does this before you opened up the issue, but now it seems obvious. The inbound data is naturally a string (coming from inbound ASCII file or stdin), but would need to be explicitly cast otherwise for the JSON to render the associated inbound FIX type properly, which isn't happening.

This has some overlap with the way data dictionaries are being employed, hence to do this properly, the data dictionary would need to be consulted based on the defined type for individual tags. e.g.:

[ { tag: '1128', val: '9', type: 'STRING' },
  { tag: '9', val: '156', type: 'LENGTH' },
  { tag: '35', val: 'X', type: 'STRING' },
  { tag: '49', val: 'CME', type: 'STRING' },
  { tag: '34', val: '431875', type: 'SEQNUM' },
  { tag: '52', val: '20130715053428544', type: 'UTCTIMESTAMP' },
  { tag: '75', val: '20130715', type: 'LOCALMKTDATE' },
  { tag: '268', val: '1', type: 'NUMINGROUP' },
  { tag: '279', val: '1', type: 'CHAR' },
  { tag: '22', val: '8', type: 'STRING' },
  { tag: '48', val: '19590', type: 'STRING' },
  { tag: '83', val: '1491', type: 'INT' },
  { tag: '107', val: 'ZCN4-ZCU4', type: 'STRING' },
  { tag: '269', val: '0', type: 'CHAR' },
  { tag: '270', val: '0.75', type: 'PRICE' },
  { tag: '271', val: '7', type: 'QTY' },
  { tag: '273', val: '53428000', type: 'UTCTIMEONLY' },
  { tag: '336', val: '2', type: 'STRING' },
  { tag: '346', val: '3', type: 'INT' },
  { tag: '1023', val: '3', type: 'INT' },
  { tag: '10', val: '233', type: 'STRING' } ]

Out of curiosity, might I ask how you are using Mongo to address this?

Will update the ticket with progress including any branches created for testing.

Thanks!

Sal

[jlroo]

Hi Salsferrazza, I did notice that the dictionaries come with the data type and everything has been working great until now that I need to aggregate and sort things. I am working on mongodb so what I do is use an script to change fields to integers.

Message

{
   "ApplVerID":"FIX50SP2",
   "BodyLength":"152",
   "MsgType":"MARKETDATAINCREMENTALREFRESH",
   "SenderCompID":"CME",
   "MsgSeqNum":"988",
   "SendingTime":"20130714210134684",
   "TradeDate":"20130715",
   "NoMDEntries":"1",
   "MDEntries":[
      {
         "MDUpdateAction":"CHANGE",
         "SecurityIDSource":"EXCHANGE SYMBOL",
         "SecurityID":"17704",
         "RptSeq":"34",
         "SecurityDesc":"ESU3",
         "MDEntryType":"OFFER",
         "MDEntryPx":"167250",
         "MDEntrySize":"125",
         "MDEntryTime":"210134000",
         "TradingSessionID":"0",
         "NumberOfOrders":"10",
         "MDPriceLevel":"7",
      }
   ],
   "CheckSum":"174"
}

Change "MsgSeqNum" field to INT

db.futures.find().forEach(function(x) {
    x.MsgSeqNum = parseInt(x.MsgSeqNum);
    db.futures.save(x);
});

Change "MDEntryPx" nested field to INT

db.futures.find().forEach(function(x) {
    x.MDEntries.forEach(function(y) {
        y.MDEntryPx= parseInt(y.MDEntryPx);
        db.futures.save(y);
    });
    db.futures.save(x);
});

[salsferrazza]

Hi jlroo,

Would you mind checking out the branch 'dictionaries' and seeing how that works for you? From the fix2json directory you've cloned:

git checkout dictionaries && git rebase origin dictionaries 

then

./fix2json.js dict/FIX50SP2.CME.xml <my input file>

The latest patch refers to the tag's type in the specified data dictionary to see if it belongs to a set of numerics. If so, the string is converted accordingly and rendered in the output JSON as such.

If your results are positive, I can merge the changes into the main branch and then republish the npm.

Thanks!

Sal

[jlroo]

Sal, Thank you for the update it works great. Now I get numeric data type. See Below:

{
   "ApplVerID":"FIX50SP2",
   "BodyLength":261,
   "MsgType":"MARKETDATAINCREMENTALREFRESH",
   "SenderCompID":"CME",
   "MsgSeqNum":934385,
   "SendingTime":"20130715145033511",
   "TradeDate":"20130715",
   "NoMDEntries":2,
   "MDEntries":[
      {
         "MDUpdateAction":"CHANGE",
         "SecurityIDSource":"EXCHANGE SYMBOL",
         "SecurityID":"17704",
         "RptSeq":1280912,
         "SecurityDesc":"ESU3",
         "MDEntryType":"BID",
         "MDEntryPx":167525,
         "MDEntrySize":216,
         "MDEntryTime":"145033000",
         "TradingSessionID":"HALFDAY",
         "NumberOfOrders":45,
         "MDPriceLevel":1
      }
   ],
   "CheckSum":"003"
}

Thank you again,

-j

[salsferrazza]

fix2json 0.4.1 addresses this issue.