Processing Day/Night VIIRS images

[SeanFGIS]

Hello,

I have been using Polar2Grid to process VIIRS imagery of Northern California and have run into a weird issue. I am able to process images with a time in the evening but not images with a time in the morning.

if anyone has any insight for how to process these other images it would be very helpful.

Example.
GMODO-SVM14_npp_d20210809_t2042254_e2048040_b50699_c20231120143530776612_oeac_ops.h5 GMODO-SVM14_npp_d20210809_t0919359_e0925163_b50692_c20231120143529392643_oeac_ops.h5

I can process the above file and its corresponding files, but I cannot process the below file and its corresponding files.

and this is the case no matter what day I try.

[djhoese]

What do you expect to happen? What is actually happening? Are you getting any error messages? Could you past the generated log file somewhere for me to see?

[SeanFGIS]

Hello,

Thanks for your response, I should have been more clear.

I can use Polar2Grid to process images from VIIRS. I am uploading GIMGO_SVI01-5, GITCO, GMODO_SVM01-16, and GMTCO files.

When they have a t19+ number in the file name the code runs fine. and I get the outputs of True and False color images along with all the individual band raster files.

when the file name has a t18- it fails to fun and doesn't output the above desired files.

any insight would be helpful thank you.

[djhoese]

True color and other reflectance-based products are only valid at day time (i.e. when there is sun reflecting off the Earth). Similarly there are some products that are only valid/useful at night (temperature differences to mimic fog detection). Polar2Grid defaults to filtering these products from generation if it detects that the majority of the provided data are not day (or not night for those products). You can find more information on this by looking at the --filter-day-products and --filter-night-products flags. Setting --filter-day-products false should generate the true color and other reflectance products regardless of amount of day pixels. Hopefully I've understood the question. Let me know if I haven't.

[SeanFGIS]

Hello David / All,

I'm working on a project requiring Fire Radiative Power (FRP) or temperature data from the VIIRS instruments on the S-NPP, NOAA-20, and NOAA-21 satellites. All of the data available online I have been able to download of FRP & Temp are daily averages, but I need detailed data from as many passes as possible, ideally six times a day (for each satellite's day and night pass).

I was told Polar2Grid might be helpful to accomplish this because it allows for the processing of each pass rather than the averages. But as mentioned above, so far I have only been able to process those true/false images for the daytime passes. Could you or anyone else give me insight into whether Polar2Grid could help process either FRP or Temperature?

any help would be appreciated.

[djhoese]

I may be misunderstanding so let me know if any of this doesn't make sense:

Polar2Grid has readers for various input file formats including VIIRS SDR and a couple EDR (level 2) file formats. In our next version we plan on adding support for even more VIIRS EDR formats. One of the existing formats we support is the VIIRS EDR Active Fire algorithm output.

In general Polar2Grid is creating images from whatever data you provide to it. In the usual SDR and EDR cases these are single swaths (a portion of one orbit) and no averaging or combining of multiple time steps or orbits is done by Polar2Grid itself. So in that sense what you've been told is correct. In some EDR cases where higher levels of processing have been performed, the data you're dealing with may be a combination of multiple SDR (level 1) band data, or combination of data from other instruments (on the same satellite or other satellites or even forecast models), or in some cases averages over time. These steps though are happening from upstream algorithm software. As far as Polar2Grid is concerned it doesn't care. It is just taking the data arrays and putting them in an image.

Now for fire products, it isn't clear to me if you're hoping for Polar2Grid to generate the FRP or Fire Temperature information from some lower-level product (ex. VIIRS SDRs) or if you want to process the output of some algorithm software. If the latter then I'll need more information about the algorithm software that you're using the output from, but the official VIIRS EDR Active Fires products should work. If you're hoping for the former then no, Polar2Grid does not typically generate new products requiring higher levels of processing beyond various RGB combinations like the true and false color images you've seen.

[SeanFGIS]

Thank you. This has given me a lot to think over.

I will reach out in the future if I have more questions.

I really appreciate your insight.

[djhoese]

No problem. I'll close this issue as the original questions have been answered but feel free to comment here even when it is closed or to create a new issue if you'd like.

[SeanFGIS]

Hello again,

You mentioned that currently, Polar2Grid can support VIIRS EDR Active Fire algorithm outputs, and I have a few specific questions about that.

Is NOAA generating an average output for the VIIRS EDR Active Fire algorithm?
Regarding the satellite data sources, are the VIIRS S-NPP, NOAA-20, and NOAA-21 all providing this EDR file, or is it limited to certain satellites?

In addition, I would appreciate guidance on the best method for downloading this data for use with Polar2Grid. Should I access it through the NOAA CLASS website under the VIIRSI_EDR section, or do you recommend a different source?

you have been a huge help with this thank you

[kathys]

Hello Sean. You asked Is NOAA generating an average output for the VIIRS EDR Active Fire algorithm? Regarding the satellite data sources, are the VIIRS S-NPP, NOAA-20, and NOAA-21 all providing this EDR file, or is it limited to certain satellites? I am not sure what you mean. The VIIRS Active Fire EDR software produces output at a single I-Band and/or M-Band resolution Field-of-View. The EDR software supports all three JPSS satellites.

As background, Polar2Grid is supported as part of the Community Satellite Processing Package (CSPP). The main use case is to provide software for the real-time direct broadcast community (ie, those who have an antenna and get the data directly from the satellite). Since most of the software comes directly from NOAA that we package for stand alone use, the output is close or identical to what you will find at the NOAA archives. For instance, if you want to find out more information about the Active Fire EDR that Polar2Grid supports, the CSPP Active Fire installation guide is here, with links to the NOAA website and Algorithm Theoretical Basis Documents: https://bin.ssec.wisc.edu/pub/CSPP/hidden/FIRE/v2.1/CSPP_Active_Fire_Installation_Guide_v2.1.pdf.

Because CSPP's primary mission is to support direct broadcast, the file naming conventions and file formats that are tested and supported in Polar2Grid are the CSPP outputs. It will not support all of the NOAA naming conventions for example. The NOAA EDR active fires products, which can be found at NOAA CLASS under the JPSS VIIRS Products (Granule) (JPSS_GRAN) product heading are not supported in P2G at this time.

I hope this helps to answer your questions.

[SeanFGIS]

Hello,

I'm focused on obtaining daily VIIRS images to analyze past fires. For example, let us use the 2021 Antelope fire in Northern California. So far, I've processed both true and false color VIIRS daytime images using P2G.

My next goal is to process FRP or a similar metric. However, the FIRE VIIRS images from NOAA I've accessed are daily averages, not individual images. I downloaded these averaged files from Earth Explorer.

My goal is to download individual VIIRS images for each day to track fire progression more accurately, rather than relying on daily averages. are the VIIRS_EDR images from NOAA CLASS already preprocessed and ready to use? Can I use VIIRS_EDR from NOAA CLASS in P2G to produce daily image files, as I have with the true and false color images?

Thank you for any insight you can provide. you both have been so helpful and will be acknowledged and cited in any publications

[kathys]

Each VIIRS Active Fire EDR NetCDF file is created on a granule basis, just like the VIIRS SDRs. You would select the files the same way you select the VIIRS SDRs, ie, over your region of interest and time period. The files contain things like fire confidence and FRP at pixel level. However, you should make sure and understand things like the FRP I-Band product is actually taken from the M-Band resolution product as described in the ATBD and the CSPP documentation.

The EDR products are not images. They are scientific algorithm retrieval files. If you want to create images from them, you can use P2G, but you will have to change the file names to the CSPP naming conventions. If you want to do that, once you have downloaded the files, we can provide you with a script to this.

[SeanFGIS]

if you could provide me with that script I would greatly appreciate it. I am going to download and process these files tomorrow during my work day.

Thank you again!