problems with code and suggestions

[rumcode]

Suggest you try different dataset from nasdaq because the dataset you picked is rather limited in message types and so it crashes and burns.

Data output should look something like using data from 08/22/2014: [A, 7832, 0, 34383145181253, 9138132, S, 100, WLL, 810000] [F, 7845, 0, 34383145198745, 9138133, S, 100, WMW, 268800, NITE] [F, 7845, 0, 34383145201499, 9138134, S, 100, WMW, 268800, ATDF] [F, 7845, 0, 34383145204380, 9138135, S, 100, WMW, 268800, CANT] [F, 7845, 0, 34383145207174, 9138136, S, 100, WMW, 268800, TRIM] [D, 7845, 0, 34383145238037, 8467825] [D, 7845, 0, 34383145240887, 8467826] [U, 7845, 0, 34383145288744, 8780722, 9138137, 500, 210300

To interpret the time stamp (col4) divide by 1 million and use the function =TEXT(E6236/86400000,"hh:mm:ss.000") in excel to view it in milliseconds For example 34383145288744 becomes 09:33:03.145

Missing case for when the field is 2 bytes in length and is an integer. This is why you get non printable characters. Code currently treats it like a char.

Missing case for the time stamp. Code currently treats as 4 bytes Int. So it only treats 4 bytes not 6 leaving a much smaller number!

So switch statement should look something like: switch ((Integer) fieldArray.get(0)) { case 1: value = (String) getChar(arr); break; case 2: value = (String) getInt(arr); break; case 3: value = (String) getString(arr, (Integer) fieldArray.get(1)); break; case 4: value = (String) getLong(arr); break; case 5: //for 2 byte integer value = (String) getShort(arr); break; case 6: // for timestamp value = (String) getArbitrayLengthNumber(arr); break;

public Object getShort(byte[] payload) { return Integer.toString(ByteBuffer.wrap(payload).getShort());
}

public Object getArbitrayLengthNumber(byte[] payload) { long value = 0; for (int i = 0; i < payload.length; i++) { value = (value << 8) + (payload[i] & 0xff); } return Long.toString(value); } Problems in your yaml file: Missing message "N" which is the Retail Price Improvement indicator Section 4.7 You need this because the datasets are before it was discontinued. Program crashes and burns.

For message type ="K" you are missing the field tracking Number

The additional corrections to the yaml file are as follows (it's ok to leave the decimal changes out and it will run. )

formats:

• MessageType : [1,1]ok
• Timestamp : [2,6] should be integer length 6
• StockLocate : [1,2] should be integer length 2
• TrackingNumber : [1,2] should be integer length 2
• Stock : [3,8] ok
• Shares : [2,4] ok
• Price : [2,4] length is 4 but has 4 decimal places
• BuySellIndicator : [1,1]ok
• OrderReferenceNumber : [4,8]ok
• MatchNumber : [4,8]ok
• Attribution : [3,4]ok
• CanceledShares : [2,4]ok
• CrossPrice : [2,4] length is 4 but has 4 decimal places
• CrossShares : [4,8]ok
• CrossType : [1,1]ok
• CurrentReferencePrice : [2,4] length is 4, but has 4 decimal places
• EventCode : [1,1] ok
• ExecutedShares : [2,4] ok
• ExecutionPrice : [2,4] length is 4, but has 4 decimal places
• FarPrice : [2,4] length is 4, but has 4 decimal places
• FinancialStatusIndicator : [1,1]ok
• Imbalance : [4,8]ok
• ImbalanceDirection : [1,1]ok
• MPID : [3,4]ok
• MarketCatagory : [1,1]ok
• MarketMakerMode : [1,1]ok
• MarketParticipantState : [1,1]ok
• NearPrice : [2,4]length is 4, but has 4 decimal places
• NewOrderReferenceNumber : [4,8]ok
• OriginalOrderReferenceNumber : [4,8]ok
• PairedShares : [4,8]ok
• PriceVariationIndicator : [1,1]ok
• PrimaryMarketMaker : [1,1]ok
• Printable : [1,1]ok
• Reason : [3,4]ok
• RegSHOAction : [1,1]ok
• Reserved : [1,1]ok
• RoundLotSize : [2,4]ok
• RoundLotsOnly : [1,1]ok
• TradingState : [1,1]ok
• IssueClassification : [1,1]ok
• IssueSubType : [1,2]incorrect alpha two bytes length
• Authenticity : [1,1]ok
• SSThresholdIndicator : [1,1]ok
• IPOFlag : [1,1]ok
• LULD : [1,1]ok
• ETPFlag : [1,1]ok
• ETPFactor : [1,4]incorrect integer 4 bytes length
• InverseIndicator : [1,1]ok
• LocateCode : [1,2]incorrect integer 2 bytes length
• Level1 : [3,8]length is 8, but has 8 decimal places
• Level2 : [3,8]length is 8, but has 8 decimal places
• Level3 : [3,8]length is 8, but has 8 decimal places
• BreachedLevel : [1,1]ok
• IPOQuotRelTime : [1,4]incorrect integer 4 bytes length
• IPOQuotRelQual : [1,1]ok
• IPOPrice : [1,4] integer length is 4, but has 4 decimal places

[Amay22]

@rumcode I couldn't find NASDAQ ITCH data for the date : 08/22/2014 I tried to run my simulation again and corrected the itch5 yaml file as you suggested. But I am not able to test the validity of the data. Is the data correct. Please use the updated itch5.yaml and parsers.java and tell me if you are getting the correct data. I have just realized that this code is way to heavy and I have tried cleaning it up a bit but my laptop burns up. I'll write a ruby script that will run it in parts.

[Amay22]

@rumcode I dont know which data you have showed in your comments but I most certainly didnt get any stock price equal to 810000. Actually I didn't even get that timestamp or a comany called WLL around the timestamp. I used the 08/02/2014 data.

[A, 7832, 0, 34383145181253, 9138132, S, 100, WLL, 810000]

[rumcode]

It's my mistake that I reported the wrong date in my when I noted the data sample.

However, I would recommend that you change the data that you suggest people run with your code. If you look at the calendar you will see that 8/2/2014 is a Saturday. It's a small data set. Not sure why Nasdaq posted this data. Below is a cut and paste of historical data from Yahoo finance for the company Whiting Petroleum (WLL) . As you can see there is no 8/2/2014.

Aug 7, 2014 86.71 86.92 83.90 84.47 1,782,500 84.47 Aug 6, 2014 85.08 88.26 84.81 86.23 1,595,700 86.23 Aug 5, 2014 87.33 87.72 84.18 86.00 2,549,900 86.00 Aug 4, 2014 84.70 88.47 83.85 88.03 2,537,000 88.03 Aug 1, 2014 87.88 87.88 83.48 84.40 3,462,800 84.40 Jul 31, 2014 88.99 90.65 86.23 88.49 4,163,600 88.49