Exception in thread "main" org.joda.time.IllegalFieldValueException: Value 15 for monthOfYear must be in the range [1,12]

[casa-henrym]

I'm not able to get the J.P. Instruments software installed (in the case of EzTrends - does not detect SQL Server Compact 4.0) or working (in the case of EzSave) so I tried to extract my data with this tool.

This is the exception I receive:

Exception in thread "main" org.joda.time.IllegalFieldValueException: Value 15 for monthOfYear must be in the range [1,12]
        at org.joda.time.field.FieldUtils.verifyValueBounds(FieldUtils.java:252)
        at org.joda.time.chrono.BasicChronology.getDateMidnightMillis(BasicChronology.java:612)
        at org.joda.time.chrono.BasicChronology.getDateTimeMillis(BasicChronology.java:177)
        at org.joda.time.chrono.AssembledChronology.getDateTimeMillis(AssembledChronology.java:133)
        at org.joda.time.chrono.ZonedChronology.getDateTimeMillis(ZonedChronology.java:118)
        at org.joda.time.chrono.AssembledChronology.getDateTimeMillis(AssembledChronology.java:133)
        at org.joda.time.base.BaseDateTime.<init>(BaseDateTime.java:258)
        at org.joda.time.base.BaseDateTime.<init>(BaseDateTime.java:199)
        at org.joda.time.DateTime.<init>(DateTime.java:397)
        at edmtools.FlightParser.parseUnixTimestamp(Unknown Source)
        at edmtools.FlightParser.parseFlightHeader(Unknown Source)
        at edmtools.FlightParser.parseHeaderAndSkipData(Unknown Source)
        at edmtools.JpiDecoder.decode(Unknown Source)
        at edmtools.tools.JpiDecode.run(Unknown Source)
        at edmtools.tools.CommandLineTool.initAndRun(Unknown Source)
        at edmtools.tools.JpiDecode.main(Unknown Source)

It could be that my files are corrupted but I suspect it may be because the file format has changed. These files were generated by a number of EDM 830s. Funnily enough the EzSave software complains about the date too:

Here's the relevant bit of code:

  private long parseUnixTimestamp(int packedDate, int packedTime) {
    int year = (packedDate & 0xfe00) >> 9;
    year += (year >= 75) ? 1900 : 2000;
    DateTime dateTime = new DateTime(
        year,
        (packedDate & 0x01e0) >> 5,
        packedDate & 0x001f,
        (packedTime & 0xf800) >> 11,
        (packedTime & 0x07e0) >> 5,
        (packedTime & 0x001f) * 2);
    return dateTime.getMillis() / 1000;
  }

Could it be that the mask or the bitshift have changed? Or that packedDate and packedTime are at a different offset within the binary data?

[wannamak]

It could be that the encoding has changed. I have not tried to use the tool to decode EDM830 data. Keith

On Mon, Dec 21, 2020 at 1:31 AM Michael Henry notifications@github.com wrote:

I'm not able to get the J.P. Instruments software installed (in the case of EzTrends - does not detect SQL Server Compact 4.0) or working (in the case of EzSave) so I tried to extract my data with this tool.

This is the exception I receive:

Exception in thread "main" org.joda.time.IllegalFieldValueException: Value 15 for monthOfYear must be in the range [1,12] at org.joda.time.field.FieldUtils.verifyValueBounds(FieldUtils.java:252) at org.joda.time.chrono.BasicChronology.getDateMidnightMillis(BasicChronology.java:612) at org.joda.time.chrono.BasicChronology.getDateTimeMillis(BasicChronology.java:177) at org.joda.time.chrono.AssembledChronology.getDateTimeMillis(AssembledChronology.java:133) at org.joda.time.chrono.ZonedChronology.getDateTimeMillis(ZonedChronology.java:118) at org.joda.time.chrono.AssembledChronology.getDateTimeMillis(AssembledChronology.java:133) at org.joda.time.base.BaseDateTime.(BaseDateTime.java:258) at org.joda.time.base.BaseDateTime.(BaseDateTime.java:199) at org.joda.time.DateTime.(DateTime.java:397) at edmtools.FlightParser.parseUnixTimestamp(Unknown Source) at edmtools.FlightParser.parseFlightHeader(Unknown Source) at edmtools.FlightParser.parseHeaderAndSkipData(Unknown Source) at edmtools.JpiDecoder.decode(Unknown Source) at edmtools.tools.JpiDecode.run(Unknown Source) at edmtools.tools.CommandLineTool.initAndRun(Unknown Source) at edmtools.tools.JpiDecode.main(Unknown Source)

It could be that my files are corrupted but I suspect it may be because the file format has changed. These files were generated by a number of EDM 830s. Funnily enough the EzSave software complains about the date too:

[image: EzSave] https://user-images.githubusercontent.com/19833540/102746034-25f0bc80-43a9-11eb-9fcc-724d63d61999.png

Here's the relevant bit of code:

private long parseUnixTimestamp(int packedDate, int packedTime) { int year = (packedDate & 0xfe00) >> 9; year += (year >= 75) ? 1900 : 2000; DateTime dateTime = new DateTime( year, (packedDate & 0x01e0) >> 5, packedDate & 0x001f, (packedTime & 0xf800) >> 11, (packedTime & 0x07e0) >> 5, (packedTime & 0x001f) * 2); return dateTime.getMillis() / 1000; }

Could it be that the mask or the bitshift have changed? Or that packedDate and packedTime are at a different offset within the binary data?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/wannamak/edmtools/issues/1, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADDNK75P7EKMP2NFIZUM6DSV3TUHANCNFSM4VDYW7ZQ .

[casa-henrym]

Assuming the date struct defined at https://www.peter2000.co.uk/aviation/jpi/jpihackcpp.txt is correct:

// pack a date into 16 bits
struct datebits {
   unsigned day:5;
   unsigned mon:4;
   unsinged year:7;
};

...then the data I'm looking for that would generate the date presented in the screenshot above (Year = 127, Month = 15, Day = 21) would be: 11111111 11110101 and I can find that sequence at the end of the third line (bytes 17 and 18):

$ xxd -b -s 1208 -l 30 U190506.JPI
00004b8: 00000000 00001000 01111000 00111101 01011110 00000000  ..x=^.
00004be: 00000100 00000000 01000000 11100010 00000000 01111000  ..@..x
00004c4: 00000000 00000011 10101101 00000100 11111111 11110101  ......
00004ca: 00110101 00011100 00000000 01101100 00000000 00000110  5..l..
00004d0: 00100101 10001100 00000010 11010000 01011001 00000000  %...Y.

Bytes 17 and 18 happen to be where edmtools looks for the date if hasExtraFlightHeaderConfiguration() and isBuildNumberAtLeast(880) return true. The exception text confirms that edmtools has identified 15 as the month value.

I don't know when the flight actually occurred, but it is the first of 60 flights in a file that was extracted in May 2019, so 12 Dec 2018 is perfectly reasonable: 00100101 10001100, found at bytes 25 and 26. This happens to be preceded by a byte containing 6, which is the expected interval in seconds. So it seems the format has changed for the EDM 830, but J.P. Instruments hasn't updated EzSave to accommodate the new format and I cannot get EzTrends - which may have been updated - to work.

For completeness: the first two bytes correctly represent the flight number: "8". The second pair of bytes matches the value for low flags in the $C header ($C,830,30781,32272,1024,17634,120,140,2010,0*5A). The third pair of bytes should be the high flags, which does not match my $C header. From that point until the record interval, in bytes 23 and 24, I don't recognise any of the fields.

It's been a fun journey, but I've already spent too much time on this. I might just quickly hack the source to skip bytes and start reading the record interval at byte 23. If there have been other changes to the format then something else will bring me undone, but I might be able to get some data this way.

[casa-henrym]

My quick hack showed promise... briefly. After this line in FlightParser.java:

    @SuppressWarnings("unused")
    int unknown = inputStream.readWord();

...I appended four more calls to readWord() to get me up to byte 23, which permitted me to list the contents of the JPI files:

$ scripts/JpiDecode.sh --list ~/data/U190506.JPI
Flight number    8 at 2018-12-12T00:22:32.000+11:00
Flight number    9 at 2019-04-11T00:27:06.000+10:00
Flight number   10 at 2019-04-11T00:37:26.000+10:00
Flight number   11 at 2019-04-11T00:58:46.000+10:00
...

However, my attempt to decode the files produced an exception:

Exception in thread "main" java.lang.NullPointerException
        at edmtools.DataRecordParser.updateProtoValue(DataRecordParser.java:96)
        at edmtools.DataRecordParser.parse(DataRecordParser.java:79)
        at edmtools.FlightParser.parseFlightData(FlightParser.java:167)
        at edmtools.FlightParser.parse(FlightParser.java:62)
        at edmtools.JpiDecoder.decode(JpiDecoder.java:104)
        at edmtools.tools.JpiDecode.run(JpiDecode.java:64)
        at edmtools.tools.CommandLineTool.initAndRun(CommandLineTool.java:67)
        at edmtools.tools.JpiDecode.main(JpiDecode.java:42)

This has been fun but for now I have to move on. I might come back to this later.