deeplycloudy / glmtools

GOES-R Geostationary Lightning Mapper Tools
BSD 3-Clause "New" or "Revised" License
62 stars 33 forks source link

--split_events not returning new .nc file of events #67

Open MethaneRain opened 4 years ago

MethaneRain commented 4 years ago

I'm trying to run a simple script to get the flash extent density for some GLM files and I'm running into some bizarre behavior.

I've installed the glm via https://github.com/deeplycloudy/glmtools/blob/master/docs/index.rst with no installation errors, but when I try running the glm_test_data_new_grid_dev.ipynb I can't get the separate event .nc files that are needed.

From the example notebook:

import subprocess
import os, glob
import tempfile
tmpdir = tempfile.TemporaryDirectory()

import glmtools
from glmtools.test.common import get_sample_data_path
# glmtools_path = os.path.abspath(glmtools.__path__[0])
glmtools_path = os.path.abspath('.')

# Set the start time and duration
startdate = datetime(2018, 7, 2, 4, 30)
duration = timedelta(0, 60*5)
enddate = startdate+duration

cmd = "python {0}/grid/make_GLM_grids.py -o {1}"
cmd += " --fixed_grid --split_events --goes_position=east --goes_sector=meso"
cmd += " --ctr_lat=33.5 --ctr_lon=-101.5 --dx=2.0 --dy=2.0"
cmd += " --start={3} --end={4} {2}"

cmd = cmd.format(glmtools_path, tmpdir.name, ' '.join(samples),
                startdate.isoformat(), enddate.isoformat())
print (cmd)
#out_bytes = subprocess.check_output(cmd.split())
# print(out_bytes.decode('utf-8'))
print()
grid_dir_base = tmpdir.name
nc_files = glob.glob(os.path.join(grid_dir_base, startdate.strftime('%Y/%b/%d'),'*.nc'))
print(nc_files)

I get back an empty nc_files list:

python /glmtools/examples/grid/make_GLM_grids.py -o 

/var/folders/c2/404k4rn50k53k7nrlzmsh3br0000gn/T/tmpg4lk6ydh --fixed_grid --split_events -

-goes_position=east --goes_sector=meso --ctr_lat=33.5 --ctr_lon=-101.5 --dx=2.0 --dy=2.0 --

start=2018-07-02T04:30:00 --end=2018-07-02T04:35:00 

/glmtools/glmtools/test/data/OR_GLM-L2-LCFA_G16_s20181830433000_e20181830433200_c20181830433231.nc 

/glmtools/glmtools/test/data/OR_GLM-L2-LCFA_G16_s20181830433200_e20181830433400_c20181830433424.nc 

/glmtools/glmtools/test/data/OR_GLM-L2-LCFA_G16_s20181830433400_e20181830434000_c20181830434029.nc

[]

I'm expecting to see several .nc files such as: ...flash_init.nc ...flash_extent.nc ...source.nc ...total_energy.nc ...footprint.nc ...group_init.nc ...group_extent.nc ...group_area.nc ...flash_area_min.nc

Any ideas on what I'm doing wrong? I can provide cat make_GLM_grid.log details if that helps.

deeplycloudy commented 4 years ago

Hi @MethaneRain, to take your last question first, the master branch now outputs a unified grid file format, where all grids files are in the same .nc file.

We also changed how the output path works. Instead of creating a default path convention, it is required to specify the exact output path you want. The new_grid_dev notebook is outdated. Try using plot_glm_test_data.ipynb instead, which should be current. I suspect the code ran and produced files, but the glob command that went to look for files was no longer looking in the correct spot.