String index out of range: 0

[Shamim1977]

I was trying it with the following sample file- http://optionsdata.baruch.cuny.edu/data1/delivery/nasdaq/S20180501-v50_sample.txt and I am calling parser class from a JFrame, it parses 2 records and faces an out of index issue-

[S, 0, 0, 10949877220447, O] [R, 1, 0, 14337433606119, A, , N, 536870912, d, N, CZ, , P, N, , 1, 1308622848, ] Exception in thread "Thread-2" java.lang.StringIndexOutOfBoundsException: String index out of range: 0 at java.lang.String.charAt(String.java:658) at nasdaqitchparser.Parsers.messageIn(Parsers.java:45) at nasdaqitchparser.Parse.parse(Parse.java:65)

Can u please suggest what to do?

[Shamim1977]

...I believe, its getting stuck in the 2nd trade data.

[Amay22]

You are saying the R record is failing right? Check this config file if the data is adhering to this format: https://github.com/Amay22/NASDAQ-ITCH-5.0-Parser/blob/master/itch5.yaml#L70-L90

[Shamim1977]

Thanks for your quick reply, Amay. ...but No luck with your suggestion.

[Shamim1977]

it fails in the line "messageArr = (parsers.messageIn(payBytes));" inside the following method, under file- Parse.java payBytes is coming empty or null somehow. Even no luck with this simple handler "if (payBytes.length<=0) return null;" ...It just gets stuck.

public byte[] parse() throws IOException, InterruptedException { if (input.read() == -1) { // EOF return null; } int payLength = input.read(); byte[] payBytes = new byte[payLength]; // Get the payload

    //if (payBytes.length<=0) return null;

int offset = 0;  // Loop until we've read the full payload size
while (offset < payLength) {
    offset += input.read(payBytes, offset, payLength - offset);
}

    // Check if we are parsing and printing
    if (parseNPrint) {
        ArrayList<String> messageArr = null;
        // Send payBytes byte[] to Parsers for processing
        // An arraylist will be returned with the message
        messageArr = (parsers.messageIn(payBytes));
        System.out.println(messageArr);
    }
    return payBytes;
}

[Shamim1977]

here is a small method which I am calling from a thread to run your Parse.java script-

private void StartParsing(){ String PathSep = System.getProperty("file.separator"); String ChosenFile = this.txtITCHFileLoc.getText().trim(); String yaml_loc = new File("").getAbsolutePath()+ PathSep + "src" + PathSep

• "nasdaqitchparser" + PathSep + "itch5.yaml";

  Parse parse = new Parse(ChosenFile, yaml_loc, true);
  try {
      while (parse.parse() != null) {
          //we can add some code
      }
  } catch (IOException ex) {
      Logger.getLogger(ParsingUI.class.getName()).log(Level.SEVERE, null, ex);
  } catch (InterruptedException ex) {
      Logger.getLogger(ParsingUI.class.getName()).log(Level.SEVERE, null, ex);
  }
  
  //SetInitStat(); 

[Shamim1977]

Amay, pls help

[Amay22]

@Shamim1977 I am unable to reproduce the issue. I cloned this repo and downloaded the file and ran the following commands and it gave me a lot of parsed data.

cd ~
git clone https://github.com/Amay22/NASDAQ-ITCH-5.0-Parser
cd NASDAQ-ITCH-5.0-Parser/src
javac Parse.java
java Parse <path to S20180501-v50_sample.txt> 

Output:

There's a lot more above this output also. This file fails as it is not correct after a certain point. If you get any of the NASDAQ data and run it on a linux system with at-least 8 GB RAM then it runs for 5 and 1/2 mins for a 1 GB file and prints out everything.

[A, 7763, 0, 33284363141061, 3519184, B, 100, UBRT, 249.39]
[D, 7763, 0, 33284363156141, 3519040]
[A, 7763, 0, 33284363258237, 3519188, B, 100, UBRT, 249.42]
[D, 7763, 0, 33284363280947, 3519184]
[D, 7763, 0, 33284363315597, 3519172]
[A, 7763, 0, 33284363328086, 3519192, B, 100, UBRT, 249.43]
[D, 7763, 0, 33284363367257, 3519164]
[D, 1869, 0, 33284363750039, 3999705]
[D, 7763, 0, 33284363754714, 3519064]
[D, 7763, 0, 33284363757820, 3519076]
[A, 7763, 0, 33284363782116, 3519200, B, 100, UBRT, 249.44]
[U, 7763, 0, 33284363820717, 3519180, 3519204, 100, 250.62]
[D, 7763, 0, 33284363822825, 3519188]
[A, 7763, 0, 33284363974892, 3519208, B, 100, UBRT, 249.45]
[D, 7763, 0, 33284364024090, 3519200]
[A, 7763, 0, 33284364025086, 3519212, B, 100, UBRT, 249.46]
[D, 7763, 0, 33284364027963, 3519192]
[A, 5681, 0, 33284364972217, 3712903, S, 2700, OILD, 25.74]
[A, 5683, 0, 33284364973375, 3712907, B, 1300, OILU, 52.88]
[U, 5681, 0, 33284365012785, 3712887, 3712919, 100, 25.62]
[U, 5683, 0, 33284365026908, 3712867, 3712923, 100, 53.15]
[U, 5683, 0, 33284365034632, 3712807, 3712927, 100, 52.66]
[A, 7929, 0, 33284365080781, 3519216, B, 400, UWT, 34.37]
[A, 2049, 0, 33284365088753, 3999757, S, 2200, DNO, 49.12]
[A, 7775, 0, 33284365113885, 3519220, B, 1000, UCO, 30.18]
[U, 7929, 0, 33284365119572, 3519092, 3519224, 100, 34.24]
[A, 7929, 0, 33284365208956, 3519228, B, 1900, UWT, 34.39]
[A, 5683, 0, 33284365409073, 3712931, B, 1300, OILU, 52.89]
[A, 2172, 0, 33284365411349, 3999761, S, 8200, DWT, 8.58]
[U, 5683, 0, 33284365450848, 3712927, 3712939, 100, 52.67]
[P, 7602, 2, 33284371348395, 0, B, 100, TPR, 46.64, 31913]
[D, 950, 0, 33284373431588, 3993441]
[A, 950, 0, 33284373549024, 3999765, B, 100, BP, 44.94]
[D, 7929, 0, 33284393154321, 3519228]
Exception in thread "main" java.lang.IndexOutOfBoundsException
    at java.io.FileInputStream.readBytes(Native Method)
    at java.io.FileInputStream.read(FileInputStream.java:255)
    at Parse.parse(Parse.java:53)
    at Parse.main(Parse.java:93)

[Shamim1977]

You are right, Amay. I had the same assumption that the input file was not complete. Your Parse class nicely works with correct file. Thanks for all your time.