NOAA-GFDL / fre-cli

Python-based command line interface for FRE (FMS Runtime Environment) to compile and run FMS-based models and post-process their output.
GNU Lesser General Public License v3.0
3 stars 11 forks source link

#183 Add optional fre catalog builder --slow option (default off) #184

Closed ceblanton closed 3 weeks ago

ceblanton commented 1 month ago

Describe your changes

Add optional fre catalog builder --slow option (default off)

Issue ticket number and link (if applicable)

183

Checklist before requesting a review

ceblanton commented 1 month ago

As expected, it's slow. But @aradhakrishnanGFDL indicated it's not optimized at all yet, so there's plenty of potential to improve the performance.

an104:~/git/CatalogBuilder%>fre catalog builder --config catalogbuilder/scripts/configs/config-template.yaml --slow


Slow option turned on.. lets open some files using xarray and lookup atts aerosol_cmip.000201-000212.mmrso4.nc
/nbhome/fms/conda/envs/fre-cli/lib/python3.9/site-packages/xarray/coding/times.py:995: SerializationWarning: Unable to decode time axis into full numpy.datetime64 objects, continuing using cftime.datetime objects instead, reason: dates out of range
  dtype = _decode_cf_datetime_dtype(data, units, calendar, self.use_cftime)
standard_name found mass_fraction_of_sulfate_dry_aerosol_particles_in_air
Slow option turned on.. lets open some files using xarray and lookup atts aerosol_cmip.000201-000212.mmrsoa.nc
/nbhome/fms/conda/envs/fre-cli/lib/python3.9/site-packages/xarray/coding/times.py:995: SerializationWarning: Unable to decode time axis into full numpy.datetime64 objects, continuing using cftime.datetime objects instead, reason: dates out of range
  dtype = _decode_cf_datetime_dtype(data, units, calendar, self.use_cftime)
standard_name found mass_fraction_of_NULL_dry_aerosol_particles_in_air
Slow option turned on.. lets open some files using xarray and lookup atts aerosol_cmip.000201-000212.mmrss.n
ceblanton commented 1 month ago

@Ciheim this PR might help your evaluation of the performance of the opening-netcdf-file route for retrieving vocabulary