Decode And Handle Unknown Records

[danielbarry]

Reproduction

https://github.com/danielbarry/OpenView/commit/ea5f7e5afa2dc979aa0a41e2bdc5bfb04d4be6fa

Expected

Data is decoded and used somehow.

Actual

No idea what this data does at all. The original decoder doesn't handle the data, bit instead just skips it: https://github.com/danielbarry/OpenView/blob/master/archive/software/cdcomp.c

[danielbarry]

For the first skip we have the following clue:

/*********************************************************************/
/*                                                                   */
/* process the image histogram                                       */
/*                                                                   */
/*********************************************************************/

/* need to know record_bytes,hist_count,hist_item_type,item_count.*/
   total_bytes = 0;
   length = read_var((char *)hist,host);
   total_bytes = total_bytes + length;
   length = read_var((char *)hist+836,host);
   total_bytes = total_bytes + length;

   if (host == 2 || host == 5)             /* If non-byte swapped    */
     for (i=0;i<256;i++)                   /* host, swap bytes in    */
       hist[i] = swap_long(hist[i]);       /* the output histogram   */

   if (output_format == 1)
     {
      fwrite((char *)hist,    836,1,outfile);
      fwrite((char *)hist+836,length,1,outfile);

      /*  pad out the histogram to a multiple of RECORD_BYTES */
      for (i=total_bytes;i<RECORD_BYTES*2;i++) fputc(blank,outfile);
     }

In the Java program it tells us that it is 1024 bytes. swap_long() assumes 4 bytes and it loops over 256 of these longs, so it is at least consistent. It also is only written if output_format == 1, which is the PDS format - which seems to be easily confused with the popular "PSD" format for professional photography.

I highly suspect this is data for either histogram equalization or histogram matching - either would be very important for NASA.

[danielbarry]

For the second skip we have the following clue:

/*********************************************************************/
/*                                                                   */
/* process the engineering summary                                   */
/*                                                                   */
/*********************************************************************/

   total_bytes = 0;
   length = read_var(ibuf,host);

   if (output_format == 1)
     {
      fwrite(ibuf,length,1,outfile);
      total_bytes = total_bytes + length;

      /*  pad out engineering to multiple of 836 */
      for (i=total_bytes;i<RECORD_BYTES;i++) fputc(blank,outfile);
     }

In the Java program we are told this is 242 bytes. This will be much more difficult to figure out, all we know is that the PDS format again is interested in this information.

In a hex editor we see some potentially interesting strings:

• *MOS5.3DD1MI1188TF2116500381008F
• 0815S1+000

I suspect these are something to do with the state, possibly some sensor readings or hardware values.

[danielbarry]

For the first skip we have the following clue:

/*********************************************************************/
/*                                                                   */
/* process the image histogram                                       */
/*                                                                   */
/*********************************************************************/

/* need to know record_bytes,hist_count,hist_item_type,item_count.*/
   total_bytes = 0;
   length = read_var((char *)hist,host);
   total_bytes = total_bytes + length;
   length = read_var((char *)hist+836,host);
   total_bytes = total_bytes + length;

   if (host == 2 || host == 5)             /* If non-byte swapped    */
     for (i=0;i<256;i++)                   /* host, swap bytes in    */
       hist[i] = swap_long(hist[i]);       /* the output histogram   */

   if (output_format == 1)
     {
      fwrite((char *)hist,    836,1,outfile);
      fwrite((char *)hist+836,length,1,outfile);

      /*  pad out the histogram to a multiple of RECORD_BYTES */
      for (i=total_bytes;i<RECORD_BYTES*2;i++) fputc(blank,outfile);
     }

In the Java program it tells us that it is 1024 bytes. swap_long() assumes 4 bytes and it loops over 256 of these longs, so it is at least consistent. It also is only written if output_format == 1, which is the PDS format - which seems to be easily confused with the popular "PSD" format for professional photography.

I highly suspect this is data for either histogram equalization or histogram matching - either would be very important for NASA.

Boringly, this is just a histogram of the original image. On the other hand, it could possibly be helpful for recovering from corruption...