search

JCSDA-internal / ioda-converters

Various converters for getting obs data in and out of IODA
9 stars 4 forks source link

Converter of TOA reflectance from VIIRS/JPSS1 Moderate Resolution 6-Min L1B Swath 750m #1477

Closed weiwilliam closed 6 months ago

weiwilliam commented 7 months ago

Description

This PR adds the converter of NASA's VIIRS L1b data. It combines the two inputs of observation and geolocation. Observation data (VJ102MOD): https://ladsweb.modaps.eosdis.nasa.gov/missions-and-measurements/products/VJ102MOD Geolocation data (VJ103MOD): https://ladsweb.modaps.eosdis.nasa.gov/missions-and-measurements/products/VJ103MOD NASA's VIIRS L1b User Guide is here.

The converter has passed the coding norm test. The test of ioda-validate.x returns no error with the testoutput.

Dependencies

No dependencies

Impact

No impacts

Checklist

weiwilliam commented 7 months ago

Geometry variables are not converted, I will add it in later commits.

weiwilliam commented 7 months ago

Hi @BenjaminRuston @jeromebarre @PatNichols, I think this PR is ready for review.

BenjaminRuston commented 7 months ago

@PatNichols please check conventions

jeromebarre commented 7 months ago

@weiwilliam the coding norms test is failing 1/336 Test #1: iodaconv_compo_coding_norms .....................................................***Failed 0.62 sec

weiwilliam commented 7 months ago

@weiwilliam the coding norms test is failing 1/336 Test #1: iodaconv_compo_coding_norms .....................................................***Failed 0.62 sec

Let me check, I did use the pycodestyle to see whether it is good. There are whitespaces on those lines I just added.

PatNichols commented 7 months ago

@weiwilliam is there a ctest for you code? I am not seeing it...

PatNichols commented 7 months ago

@BenjaminRuston @gthompsnJCSDA This is resuls for the ioda_validate on the output file:

[[0m[[1m[[34m Warning: Attribute 'Location' is present but is not listed as a required or optional attribute in the spec. [[0m[[1m[[34m Warning: Attribute 'Channel' is present but is not listed as a required or optional attribute in the spec. [[0m[[1m[[34m Warning: Attribute 'Location' does not have a YAML spec. [[0m[[1m[[34m Warning: Attribute 'Channel' does not have a YAML spec.

weiwilliam commented 7 months ago

@weiwilliam is there a ctest for you code? I am not seeing it...

Oh yeah, I forgot this. I keep thinking of tests in ufo is not there yet and forgot in here. Let me work on it.

gthompsnJCSDA commented 7 months ago

@weiwilliam Do you want my assistance with this python script? I can see some things to change.

weiwilliam commented 7 months ago

@weiwilliam Do you want my assistance with this python script? I can see some things to change.

@gthompsnJCSDA Thank you, can you point me to changes you would like to have? Then I can learn more from it.

weiwilliam commented 7 months ago

Thank you, @gthompsnJCSDA! I'm testing the multiple inputs and I'll add the ctest stuff when we have decision on the folder arrangement.

weiwilliam commented 6 months ago

The coding norms are still failing:

Start   1: iodaconv_compo_coding_norms
--
1620 | 1/336 Test   #1: iodaconv_compo_coding_norms .....................................................***Failed    0.62 sec

Run the test like so: ctest -VV -R iodaconv_compo_coding_norms and look at the lines where it complains. Thanks!

@jeromebarre Yeah, I fixed the coding norms but have not pushed back yet. I'm adding the code to skip the nighttime files.

weiwilliam commented 6 months ago

Checks are passed. Here is the last warning from ioda-validate.x Warning: Attribute 'description' is present but is not listed as a required or optional attribute in the spec. @gthompsnJCSDA told me he would like to add the 'description' attribute to IODA convention.

gthompsnJCSDA commented 6 months ago

@jeromebarre Perhaps we should move ahead with this converter in the compo directory for now and change up the directory structure in some way at a future time. While it isn't ideal to have a vis satellite image converter specifically within compo folder, the alternate of creating a VIIRS folder then makes it odd to find the previous composition-type data for VIIRS in compo instead of VIIRS. What do you think about "kicking the can down the road" and dealing with re-organization later.