Update validate with additional data type support for image transformations

[jordanpadams]

• IEEE754LSB* types
• SignedLSB[4|8] types
• UnsignedLSB[4|8] types
• SignedMSB[4|8] types
• UnsignedMSB[4|8] types

---

Triage Notes We will probably need to ping IMG or other nodes for some data with example data products. This issue was created in 2016 so it may even be supported currently by Validate / PDS4 JParser

[jordanpadams]

More details from SW-17:

Here is a maven label segment for the 3rd object in mvn_sep_l2_s1-cal-svy-full_20141115_v03_r01.xml:

epoch epoch 43394 1 Last Index Fastest Time, middle of sample, in TT2000 time base SignedMSB8 ns time 2629 1

Here is a listing from my object list program that uses the tools library:

C:\usr\local\pds4-tools-0.5.0>java -cp ".;lib*" PDS4ObjectListNew mvn_sep_l2_s1-cal-svy-full_20141115_v03_r01edited.xml Open File mvn_sep_l2_s1-cal-svy-full_20141115_v03_r01edited.xml Product is PRODUCT_OBSERVATIONAL File contains 42 objects Generic Object size = 4325264 Object offset = 0 Object 1 is a generic object. Object file name = C:\usr\local\pds4-tools-0.5.0\mvn_sep_l2_s1-cal-svy-full_20141115_v03_r01.cdf print generic byte 0 j= 205 c= ═ print generic byte 1 j= 243 c= ≤ print generic byte 2 j= 0 c= print generic byte 3 j= 1 c=
print generic byte 4 j= 0 c= print generic byte 5 j= 0 c= print generic byte 6 j= 255 c=   print generic byte 7 j= 255 c=   print generic byte 8 j= 0 c= print generic byte 9 j= 0 c= Object 2 is an Array Object Is this an image: false Object size = 21032 Object offset = 21004 Object file name = C:\usr\local\pds4-tools-0.5.0\mvn_sep_l2_s1-cal-svy-full_20141115_v03_r01.cdf Array element size is 8 Axes = 1 Axes 1 offset 21004 offsetunit byte datatype IEEE754MSBDouble bits 64

axis 0 seqnbr 1 axis 0 name time axis 0 elements 2629 axis 0 localid null

i = 0 value = 1.416009657056529E9 i = 1 value = 1.416009689056529E9 Object 3 is an Array Object Is this an image: false Object size = 21032 Object offset = 43394 Object file name = C:\usr\local\pds4-tools-0.5.0\mvn_sep_l2_s1-cal-svy-full_20141115_v03_r01.cdf Array element size is 8 Axes = 1 No enum constant gov.nasa.pds.objectAccess.DataType.NumericDataType.SignedMSB8

C:\usr\local\pds4-tools-0.5.0>

I can work around this to validate these peer review files, but it would be nice to get it fixed in the next release.

I am using an edited label file because there is another error in the last object in these labels that results in a computed file size that is greater than the file size when getObject is called and gives a "Channel not open for writing - cannot extend file to required size" message. It would be nice if there were a more appropriate message.

[al-niessner]

@jordanpadams

Need sample product here as well. Since it is 7 years old and if no longer an issue from users perspective then maybe just close it?

[jordanpadams]

@rchenatjpl do we have any examples of these you know of? Or should I reach out to users?

[jordanpadams]

@al-niessner do we need more test data here?

[al-niessner]

@jordanpadams @rchenatjpl

What I need is an XML file that validates is either expected to pass or fail. What goes along with that, images of 4 pixels etc, depends on the XML.

To me, the real question is do we have users complaining that their image data type is not being processed correctly? If no, then close this ticket. If yes, then ask them for a 4 pixel image + label (XML) demonstrating the problem.

[rchenatjpl]

Holy cow, the whole set is a lot. You want all these: ComplexMSB16 ComplexMSB8 IEEE754LSBDouble IEEE754LSBSingle IEEE754MSBDouble IEEE754MSBSingle SignedByte SignedLSB2 SignedLSB4 SignedLSB8 SignedMSB2 SignedMSB4 SignedMSB8 UnsignedByte UnsignedLSB2 UnsignedLSB4 UnsignedLSB8 UnsignedMSB2 UnsignedMSB4 UnsignedMSB8 Only as images? Or is Table_Binary ok? Or both?

[jordanpadams]

@rchenatjpl whichever is easier.

@al-niessner I'm not sure anyone has completed, but this is to make sure we are actually validating all of these data types. validate is supposed to check everything the standard specifies, including all these types, so we should check all of these to ensure the archive is accurate.

[jordanpadams]

@rchenatjpl nevermind. closing as wontfix for time being. the code supports this, and works as-is with test particles throughout the system