Plot true color by using VIIRS SDR

[xigrug]

I have download some NPP VIIRS SDR data from https://www.bou.class.noaa.gov/saa/products/welcome , like： GIMGO-SVI01-SVI02-SVI03-SVI04-SVI05_npp_d20170804_t0513438_e0519241_b29891_c20171225131018848734_nobc_ops.h5 GITCO_npp_d20170804_t0513438_e0519241_b29891_c20170804111925652787_nobc_ops.h5 GMODO-SVM01-SVM02-SVM03-SVM04-SVM05-SVM06-SVM07-SVM08-SVM09-SVM10-SVM11-SVM12-SVM13-SVM14-SVM15- SVM16_npp_d20170804_t0513438_e0519241_b29891_c20171225131016538639_nobc_ops.h5

I want to plot it by satpy with https://nbviewer.jupyter.org/github/pytroll/pytroll-examples/blob/master/satpy/satpy_rayleigh_iband_enhanced.ipynb. But the file maybe not work.

[mraspaud]

@xigrug Thanks for reporting the issue. Next time please follow the template for reporting an issue, so that we can have all the information to help you sort this out. Here are a few things we need to help you better:

To Reproduce Steps to reproduce your error

# Your code here

Expected behavior A clear and concise description of what you expected to happen.

Actual results Text output of actual results or error messages including full tracebacks if applicable.

Environment Info:

• OS: [e.g. OSX, Windows, Linux]
• SatPy Version: [e.g. 0.9.0]
• PyResample Version:
• Python version

[mraspaud]

also it looks like all your channels are regrouped in just a couple files, we haven't encountered this format before afaik

[xigrug]

Thanks.

Here is my code

from satpy.scene import Scene
from glob import glob
from datetime import datetime
sdrfiles = glob("/public/home/hysplit/data/npp/20170825/h5")
#Create the scene object
scene = Scene(filenames=sdrfiles, reader='viirs_sdr')
#If you want to only look at part of the scene, you can specify a start_time and an end_time:
#scene = Scene(filenames=sdrfiles, reader='viirs_sdr', start_time=datetime(2018, 2, 25, 11, 20), end_time=datetime(2018, 2, 25, 11, 26))
#Load the data for the requested RGB
scene.load(['true_color'])
#Display
scene.show('true_color')
scene.save_dataset('true_color', filename='./viirs_true_color.png')

The issue report is

Required file type 'gmtco' not found or loaded for 'm_longitude'
Required file type 'gmtco' not found or loaded for 'm_latitude'
Failed to load coordinates for 'DatasetID(name='satellite_zenith_angle', wavelength=None, resolution=742, polarization=None, calibration=None, modifiers=())'
Failed to load coordinates for 'DatasetID(name='M03', wavelength=(0.478, 0.488, 0.498), resolution=742, polarization=None, calibration='reflectance', modifiers=('sunz_corrected',))'
Failed to load coordinates for 'DatasetID(name='solar_zenith_angle', wavelength=None, resolution=742, polarization=None, calibration=None, modifiers=())'
Failed to load coordinates for 'DatasetID(name='solar_azimuth_angle', wavelength=None, resolution=742, polarization=None, calibration=None, modifiers=())'
Failed to load coordinates for 'DatasetID(name='M04', wavelength=(0.545, 0.555, 0.565), resolution=742, polarization=None, calibration='reflectance', modifiers=('sunz_corrected',))'
Failed to load coordinates for 'DatasetID(name='satellite_azimuth_angle', wavelength=None, resolution=742, polarization=None, calibration=None, modifiers=())'
Failed to load coordinates for 'DatasetID(name='M05', wavelength=(0.662, 0.672, 0.682), resolution=742, polarization=None, calibration='reflectance', modifiers=('sunz_corrected',))'
Traceback (most recent call last):
  File "satpy_rayleigh_iband_enhanced.py", line 14, in <module>
    scene.load(['true_color'])
  File "/public/home/hysplit/software/anaconda3/envs/satpy-env/lib/python3.6/site-packages/satpy/scene.py", line 567, in load
    keepables = self.compute()
  File "/public/home/hysplit/software/anaconda3/envs/satpy-env/lib/python3.6/site-packages/satpy/scene.py", line 514, in compute
    return self.read_composites(nodes)
  File "/public/home/hysplit/software/anaconda3/envs/satpy-env/lib/python3.6/site-packages/satpy/scene.py", line 488, in read_composites
    self._generate_composite(item, keepables)
  File "/public/home/hysplit/software/anaconda3/envs/satpy-env/lib/python3.6/site-packages/satpy/scene.py", line 463, in _generate_composite
    **self.info)
  File "/public/home/hysplit/software/anaconda3/envs/satpy-env/lib/python3.6/site-packages/satpy/composites/viirs.py", line 94, in __call__
    if high_res.info["area"] != low_res.info["area"]:
KeyError: 'area'

It seems that it can not find the coordinates

Expected behavior

I want to plot the true color picture.

Environment Info:

OS: Linux admin 3.10.0-693.el7.x86_64 SatPy Version: 0.8.4 PyResample Version: 1.9.3 Python version: 3.6.5

---

Here you can get the data: https://download.bou.class.noaa.gov/download/3141296505/001

[djhoese]

Note the first two lines in your series of error messages. You are missing GMTCO (terrain-corrected geolocation for M-band data) files.

[xigrug]

Yes，and I have downloaded the data like： GMTCO_npp_d20170804_t0513438_e0519241_b29891_c20170804111925652787_nobc_ops.h5. But it seems that satpy cannot load it successfully.

[djhoese]

@xigrug It is possible that what @mraspaud mentioned is the problem. Your downloaded files seem to have grouped multiple bands and geolocation in to just a few files. Typically the VIIRS SDR files we've seen have each dataset in separate files. In my experience the option to group them like you have them can be turned off on the websites used to order the data. Would it be possible for you to download the data in this separated format?

If not, this may require changes to the reader in satpy to accept these files.

[xigrug]

Thanks djhoese ! In NOAA , I cannot download the data in this separated format. Do you some somewhere to download the right format?

[djhoese]

@xigrug Using CLASS I was able to see individual filenames:

There are "Advanced Options" in your cart page where you can de-aggregate and/or include geolocation with the data files. I think you want to de-aggregate the files and NOT include the geolocation with the data files. Typically I use data that has each band and each band's geolocation as separate files. I know there is also the option of aggregating multiple granules together and that should be supported in satpy, but I can't recall the last time I tested it.

[xigrug]

OK, I will try this. If satpy can support the aggregating multiple data will be very nice!

[djhoese]

I've done this myself a couple times now with CLASS data so I'm assuming this is no longer a problem. If this is still a problem feel free to reopen this or open a new issue.