IBM / JSONata4Java

Open Source Java version of JSONata
Apache License 2.0
88 stars 36 forks source link

$toMillis('19700104170040', '[Y0001][M01][D01][H01][m01][s01]') fails with: JSONata mapping error #264

Closed czimmet closed 1 year ago

czimmet commented 1 year ago

$toMillis('19700104170040', '[Y0001][M01][D01][H01][m01][s01]') fails with

JSONata mapping error: Invalid value for DayOfMonth (valid values 1 - 28/31): 0

However, it does work on try.jsonata.org

wnm3 commented 1 year ago

The regex being generated is not interpreting the input string correctly. I'll look into this. However, please note, the value provided by try.jsonata.org (6379440000) is also incorrect as that date is 1970-03-15T20:04:00.000Z not 1970-01-04T17:00:40.000Z which is 320440000

Because we aren't parsing the input correctly, we are getting a day of month value of 0, hence the error mentioned.

wnm3 commented 1 year ago

@crshnburn The regex being created using ([0-9]+) ends up capturing 197001041 as the year and the D (days of month) ends up as zero. I was able to get the correct answer by adding the lines commented out below in the DateTimeUtils.java around line 1100 but this breaks other tests (so it is the wrong approach):

            case DECIMAL: {
                String regex = "[0-9]+";
//                if (formatSpec.mandatoryDigits > 0) {
//                  regex = "[0-9]{"+formatSpec.mandatoryDigits;
//                  if (formatSpec.optionalDigits > 0) {
//                    regex += ","+(formatSpec.mandatoryDigits+formatSpec.optionalDigits);
//                  }
//                  regex += "}";
//                }

For example, the $toMillis('2018', '[Y1]') in toMillisFunctionTest is expected to return 1514764800000...

wnm3 commented 1 year ago

@czimmet I've checked in code with the following hack:

res = generateRegex(part.component,part.integerFormat);

...

private static MatcherPart generateRegex(char component, Format formatSpec) {

...

            case DECIMAL: {
                String regex = "[0-9]+";
                switch(component) {
                  case 'Y': {
                    regex = "[0-9]{2,4}";
                    break;
                  }
                  case 'M': //
                  case 'D': //
                  case 'H': //
                  case 'h': //
                  case 'm': //
                  case 's': {
                    regex = "[0-9]{1,2}";
                    break;
                  }
                  default: {
                    break;
                  }

                }
                if (formatSpec.ordinal) {

This passes all existing tests and works for your example as well.

To try this, you can:

cd JSONata4Java
mvn clean install

Then use the ./target/JSONata4Java-2.4.1-jar-with-dependencies.jar in your classpath

Please let me know if this works for your needs and close this issue. Thanks

czimmet commented 1 year ago

Thank you! Worked perfectly.