Mystery ADD file

[sam210723]

A couple times per day a mystery ADD file (VCID 5) will be downlinked but is never be saved to disk. The demuxer will sit on VCID 5 for a couple minutes then drop the file upon changing to VCID 63.

Will need to observe in verbose mode.

[sam210723]

Every so often a single CP_PDU with APID 0 will show up. No CP_PDU is marked last in the sequence, so processing never proceeds to the TP_File stage.

[sam210723]

Mystery file name is ADD_COMSIR1_001_*.lrit

[sam210723]

This could be a CP_PDU header parsing error since the length field of the second CP_PDU (2209) is 1 greater than the continuity counter the first CP_PDU (2208).

[sam210723]

Data is not long enough for a full CP_PDU header which causes a parsing error. Final two bytes are missing (packet data field length). APID and sequence counter are correct.

[sam210723]

This is caused by the demuxer not handling CP_PDU headers that span multiple 884 byte M_PDUs. At offset 0x370 (880), there is 4 bytes in the first M_PDU and the final two in the next M_PDU.

[sam210723]

Demuxer now handles CP_PDU headers that span two M_PDUs (27e3cc0). CCSDS.CP_PDU will only parse the header if all 6 bytes are available. TP_File still isn't being output.

[sam210723]

TP_File stops 9 CP_PDUs (8190 bytes * 9) short of it's expected length.

The PNG image can still be output, just with missing bottom rows.

[sam210723]

This is also occurring with the COMSFOG image type (ADD_COMSFOG_*.lrit).

[sam210723]

Sample COMSFOG image

[sam210723]

A work-around has been put in place for short TP_Files in fab4758. All virtual channel handlers are now notified when the current VCID changes. This triggers a check for un-finished TP_Files, which are then decrypted, parsed, and written to disk. A warning is also printed to the console.

def notify(self, vcid):
    """
    Notifies virtual channel handler of change in VCID
    """

    # No longer the active channel handler  
    if vcid != self.VCID:
        # Channel has unfinished TP_File
        if self.cTPFile != None:
            # Handle S_PDU (decryption)
            spdu = CCSDS.S_PDU(self.cTPFile.PAYLOAD, self.keys)

            # Create new xRIT file
            xrit = CCSDS.xRIT(spdu.PLAINTEXT)
            xrit.save(self.outputPath)
            xrit.print_info()
            print("    FILE IS INCOMPLETE (Known issue with COMSFOG / COMSIR images)")

Console output:

[VCID 5] GK-2A: ADDITIONAL DATA
  [NEW FILE] Additional Data: "ADD_RWW3A_004_20191204_104335_00.lrit"
  [NEW FILE] Additional Data: "ADD_COMSFOG_003_20191204_104410_00.lrit"
    FILE IS INCOMPLETE (Known issue with COMSFOG / COMSIR images)
    92% OF EXPECTED LENGTH

[VCID 0] GK-2A: FULL DISK
  [NEW FILE] Image Data: "IMG_FD_064_IR105_20191204_105006_01.lrit"

[sam210723]

@usa_satcom has reported the same behaviour with his XRIT software. More confident in this being a transmission issue at the KMA end now.