search

pds-data-dictionaries / PDS4-LDD-Issue-Repo

Issue repository for tracking all PDS4 Discipline Dictionary-related issues, new feature requests, and releases.
Apache License 2.0
2 stars 1 forks source link

[ldd-img] StarPixel_Lossless_Parameters class needed #164

Closed thareUSGS closed 3 years ago

thareUSGS commented 3 years ago

Issue Type update attributes (email from Shin-ya Murakami)

Describe the issue identified (if applicable) New class and attributes to support instruments on JAXA's Akatsuki and Hayabusa2

Describe the solution you'd like

<DD_Class>
    <name>StarPixel_Lossless_Parameters</name>
    <version_id>1.0</version_id>
    <local_identifier>StarPixel_Lossless_Parameters</local_identifier>
    <submitter_name>Shin-ya Murakami</submitter_name>
    <definition>
        The StarPixel_Lossless_Parameters class contains attributes describing onboard
        compression parameters specific to StarPixel Lossless image compression.
        StarPixel Lossless is developed based on FELICS and used by the JAXA's spacecrafts,
        Akatsuki and Hayabusa2. StarPixel Lossless is a lossless compression algorithm
        and also known as HIREW.
    </definition>
    <abstract_flag>false</abstract_flag>
    <element_flag>false</element_flag>
    <DD_Association>
        <identifier_reference>starpixel_initial_subsampling_interval</identifier_reference>
        <reference_type>attribute_of</reference_type>
        <minimum_occurrences>0</minimum_occurrences>
        <maximum_occurrences>1</maximum_occurrences>
    </DD_Association>
    <DD_Association>
        <identifier_reference>Image_Compression_Segment</identifier_reference>
        <reference_type>component_of</reference_type>
        <minimum_occurrences>0</minimum_occurrences>
        <maximum_occurrences>*</maximum_occurrences>
    </DD_Association>
</DD_Class>

<DD_Class>
    <name>StarPixel_Flexible_Parameters</name>
    <version_id>1.0</version_id>
    <local_identifier>StarPixel_Flexible_Parameters</local_identifier>
    <submitter_name>Shin-ya Murakami</submitter_name>
    <definition>
        The StarPixel_Flexible_Parameters class contains attributes describing onboard
        compression parameters specific to StarPixel Flexible image compression.
        StarPixel Flexible is developed based on StarPixel Lossless and used by the JAXA's spacecraft,
        Hayabusa2. StarPixel Flexible is a lossy compression algorithm.
    </definition>
    <abstract_flag>false</abstract_flag>
    <element_flag>false</element_flag>
    <DD_Association>
        <identifier_reference>starpixel_initial_subsampling_interval</identifier_reference>
        <reference_type>attribute_of</reference_type>
        <minimum_occurrences>0</minimum_occurrences>
        <maximum_occurrences>1</maximum_occurrences>
    </DD_Association>
    <DD_Association>
        <identifier_reference>starpixel_quality</identifier_reference>
        <reference_type>attribute_of</reference_type>
        <minimum_occurrences>0</minimum_occurrences>
        <maximum_occurrences>1</maximum_occurrences>
    </DD_Association>   
    <DD_Association>
        <identifier_reference>Image_Compression_Segment</identifier_reference>
        <reference_type>component_of</reference_type>
        <minimum_occurrences>0</minimum_occurrences>
        <maximum_occurrences>*</maximum_occurrences>
    </DD_Association>
</DD_Class>

<DD_Attribute>
    <name>starpixel_initial_subsampling_interval</name>
    <version_id>1.0</version_id>
    <local_identifier>starpixel_initial_subsampling_interval</local_identifier>
    <nillable_flag>false</nillable_flag>
    <submitter_name>Shin-ya Murakami</submitter_name>
    <definition>
        The value gives exponent n for initial subsampling interval 2**n.
        See Takada+2007 (https://doi.org/10.1109/IGARSS.2007.4422835) for the detail.
    </definition>
    <DD_Value_Domain>
        <enumeration_flag>false</enumeration_flag>
        <value_data_type>ASCII_NonNegative_Integer</value_data_type>
        <minimum_value>0</minimum_value>
        <maximum_value>8</maximum_value>
    </DD_Value_Domain>
</DD_Attribute>

<DD_Attribute>
    <name>starpixel_quality</name>
    <version_id>1.0</version_id>
    <local_identifier>starpixel_quality</local_identifier>
    <nillable_flag>false</nillable_flag>
    <submitter_name>Shin-ya Murakami</submitter_name>
    <definition>
        The value is quality degradation parameter. The value of quality
        degradation parameter is higher, quality of lossy compressed image
        will be worse, and the value is lower, the quality will be fine.
        If value of quality degradation parameter is 0, there will be no
        quality degradation.
     </definition>
    <DD_Value_Domain>
        <enumeration_flag>false</enumeration_flag>
        <value_data_type>ASCII_NonNegative_Integer</value_data_type>
        <minimum_value>0</minimum_value>
        <maximum_value>32</maximum_value>
    </DD_Value_Domain>
</DD_Attribute>

<DD_Rule>
    <local_identifier>starpixel_lossless_params_child_check</local_identifier>
    <rule_context>//img:StarPixel_Lossless_Parameters</rule_context>
    <DD_Rule_Statement>
        <rule_type>Assert</rule_type>
        <rule_test>count(child::*) > 0</rule_test>
        <rule_message>IMG:error:starpixel_lossless_params_child_check: img:StarPixel_Lossless_Parameters must have at least 1 attribute or class specified.</rule_message>
    </DD_Rule_Statement>
</DD_Rule>

<DD_Rule>
    <local_identifier>starpixel_flexible_params_child_check</local_identifier>
    <rule_context>//img:StarPixel_Flexible_Parameters</rule_context>
    <DD_Rule_Statement>
        <rule_type>Assert</rule_type>
        <rule_test>count(child::*) > 0</rule_test>
        <rule_message>IMG:error:starpixel_flexible_params_child_check: img:StarPixel_Flexible_Parameters must have at least 1 attribute or class specified.</rule_message>
    </DD_Rule_Statement>
</DD_Rule>

Then I can use these classes and attributes as follows...

<img:Imaging>
   ...
   <img:Onboard_Compression>
       <img:onboard_compression_class>Lossless</img:onboard_compression_class>
       <img:onboard_compression_type>StarPixel Lossless</img:onboard_compression_type>
       <img:StarPixel_Lossless_Parameters>
           <img:starpixel_initial_subsampling_interval>3</img:starpixel_initial_subsampling_interval>
           <img:Image_Compression_Segment>
               <img:segment_number>???</img:segment_number>
               <img:first_line>1</img:first_line>
               <img:first_sample>1</img:first_sample>
               <img:lines>128</img:lines>
               <img:samples>128<img:samples>
               <img:segment_type>Corrupted</img:segment_type>
           </img:Image_Compression_Segment>
       </img:StarPixel_Lossless_Parameters>
   </img:Onboard_Compression>
   ...
</img:Imaging>

and

<img:Imaging>
   ...
   <img:Onboard_Compression>
       <img:onboard_compression_class>Lossless</img:onboard_compression_class>
       <img:onboard_compression_type>StarPixel Lossless</img:onboard_compression_type>
       <img:StarPixel_Lossless_Parameters>
           <img:starpixel_initial_subsampling_interval>3</img:starpixel_initial_subsampling_interval>
           <img:Image_Compression_Segment>
               <img:segment_number>???</img:segment_number>
               <img:first_line>1</img:first_line>
               <img:first_sample>1</img:first_sample>
               <img:lines>128</img:lines>
               <img:samples>128<img:samples>
               <img:segment_type>Corrupted</img:segment_type>
           </img:Image_Compression_Segment>
       </img:StarPixel_Lossless_Parameters>
   </img:Onboard_Compression>
   ...
</img:Imaging>

Note that we have to consider about segment_number. In our case, each tile is compressed separately, so each tile might be considered as a compression segment. We don't have a definition for one-dimensional value for identification of tile. We identify tile by a value pair, such as (0,0), which is a position in the tile coordinate, The tile position is expressed as (i,j) where i and j are 0, 1, 2, 3, 4, 5, 6, or 7. The value_data_type of segment_number attribute is ASCII_NonNegative_Integer. I'm not sure that one can code (0,0) as 00 and (3,7) as 37. It doesn't seem good....

thareUSGS commented 3 years ago

For starpixel_quality there is some concern that an increasing value means worse (more degradation). As pointed out by @rgdeen this is not typical as used here and generally with other formats (e.g. JPEG, where 1 - 100, and 100 means best).

I concur with Bob where " a higher number means higher quality (less compression)."

So how about we simply change the attribute to starpixel_degradation? I will try that out in the branch.

If you still need a quality attribute, can you use a value between 0-1 be used (which is consistent with existing attributes already in IMG). Where 1 is best.

thareUSGS commented 3 years ago

Instead of segment_type = [Corrupted, Not Corrupted].

How about simply a Boolean: segment_corrupted_flag = [True, False]

thareUSGS commented 3 years ago

Lastly, as you stated the use of segment_number is not ideal. Can that be an index into the tile's value pair (i,j)?

So for example: segment_number = 7 => tile (1, 2), using a base 0 system

[ 1, 2,  3,  4
  5, 6,  7,  8
  9, 10, 11, 12 ]
thareUSGS commented 3 years ago

test development branch pushed. These new examples (untested) assume segment_number can be an index into the tiles. https://github.com/pds-data-dictionaries/ldd-img/tree/v1860_JAXA/build/development/95a564af7bb2f3ebedf6d00bf4efd2ddac7f0366

<img:Imaging>
   ...
   <img:Onboard_Compression>
       <img:onboard_compression_class>Lossless</img:onboard_compression_class>
       <img:onboard_compression_type>StarPixel Lossless</img:onboard_compression_type>
       <img:StarPixel_Lossless_Parameters>
           <img:starpixel_initial_subsampling_interval>3</img:starpixel_initial_subsampling_interval>
           <img:Image_Compression_Segment>
               <img:segment_number>7</img:segment_number>
               <img:first_line>1</img:first_line>
               <img:first_sample>1</img:first_sample>
               <img:lines>128</img:lines>
               <img:samples>128<img:samples>
               <img:segment_corrupted_flag>True</img:segment_corrupted_flag>
           </img:Image_Compression_Segment>
       </img:StarPixel_Lossless_Parameters>
   </img:Onboard_Compression>
   ...
</img:Imaging>

Note the change to use starpixel_degradation 

<img:Imaging>
   ...
   <img:Onboard_Compression>
       <img:onboard_compression_class>Lossy</img:onboard_compression_class>
       <img:onboard_compression_type>StarPixel Flexible</img:onboard_compression_type>
       <img:StarPixel_Flexible_Parameters>
           <img:starpixel_initial_subsampling_interval>3</img:starpixel_initial_subsampling_interval>
           <img:starpixel_degradation>15</img:starpixel_degradation>
           <img:Image_Compression_Segment>
               <img:segment_number>7</img:segment_number>
               <img:first_line>1</img:first_line>
               <img:first_sample>1</img:first_sample>
               <img:lines>128</img:lines>
               <img:samples>128<img:samples>
               <img:segment_corrupted_flag>False</img:segment_corrupted_flag>  <!-- optional -->
           </img:Image_Compression_Segment>
       </img:StarPixel_Flexible_Parameters>
   </img:Onboard_Compression>
   ...
</img:Imaging>
murashinln commented 3 years ago

I saw the test development branch and it seems that StarPixel_Flexible_Parameters and StarPixel_Lossless_Parameters are placed just under the Imaging class, instead of the Onboard_Compression class. I suppose these two Parameters classes will be placed at adjacent to the other Parameters classes, ICER_Parameters, LOCO_Parameters, JPEG_Parameters, and JPEG_Progressive_Parameters. Is the current build correct?

thareUSGS commented 3 years ago

Yes good catch, update to be under Onboard_Compression as an XSChoice. I hope the "tiles' value pair into the scalar index" works for you. If not we can revisit it. Maybe getting close?

new build to check out: https://github.com/pds-data-dictionaries/ldd-img/tree/v1860_JAXA/build/development/1ce8c67ce58257c38491bb45fc2b60981cc5b6c1

murashinln commented 3 years ago

Thank you, it seems ok.

I found that wording in the definition of the starpixel_degradation attribute should be fixed:

If value of degradation parameter is 0, there will is no degradation.

should be

If value of degradation parameter is 0, there will be no degradation.

?

thareUSGS commented 3 years ago

@murashinln got it - update in branch. Please check for anything else.

murashinln commented 3 years ago

Sorry for taking long time, I updated my label using the StarPixel_Lossless_Parameters class, and checked it by the validate command. I put the value True to the segment_corrupted_flag attribute according to the description in the segment_corrupted_flag,

Valid values are "True" or "False".

but the type of the segment_corrupted_flag is specified as ASCII_Boolean, so the permissible values are true and false. Therefore the description should be

Valid values are "true" or "false".

murashinln commented 3 years ago

I forgot that, according to the Standards Reference, 1 and 0 can also be accepted. I don't think we need the description about the permissible values because the permissible values can be understood by the type, ASCII_Boolean. I think it will be good if adding next words to explicitly associate the boolean values to the cases:

If the value is true, the segment is corrupted, and if the value is false, the segment is not corrupted.

thareUSGS commented 3 years ago

@murashinln updates added. Do you think we are ready to merge into "main"?

murashinln commented 3 years ago

@thareUSGS Yes, I think so.