search

podaac / l2ss-py

Level 2 subsetter with Harmony integration
https://podaac.github.io/l2ss-py/
Apache License 2.0
11 stars 11 forks source link

Variable attributes are incorrect #182

Closed nlenssen2013 closed 8 months ago

nlenssen2013 commented 1 year ago

attributes with '/' are replaced with '_'. Units such as W/m^2 need to be conserved with the slash in the string. Coordinate attributes with slashes need to be conserved as well.

sliu008 commented 1 year ago

heres the error i get when i remove the replace '/' code, are you able to subset your collection when you maintain the '/' ?


2023-08-04 10:15:58 2023-08-04T17:15:58.596Z [debug]: ERROR:root:NetCDF: Name contains illegal characters
2023-08-04 10:15:58 Traceback (most recent call last):
2023-08-04 10:15:58   File "/home/dockeruser/.local/lib/python3.9/site-packages/harmony/cli.py", line 215, in _invoke
2023-08-04 10:15:58     (out_message, out_catalog) = adapter.invoke()
2023-08-04 10:15:58   File "/home/dockeruser/.local/lib/python3.9/site-packages/harmony/adapter.py", line 116, in invoke
2023-08-04 10:15:58     return (self.message, self._process_catalog_recursive(self.catalog))
2023-08-04 10:15:58   File "/home/dockeruser/.local/lib/python3.9/site-packages/harmony/adapter.py", line 181, in _process_catalog_recursive
2023-08-04 10:15:58     output_item = self.process_item(item.clone(), source)
2023-08-04 10:15:58   File "/home/dockeruser/.local/lib/python3.9/site-packages/podaac/subsetter/subset_harmony.py", line 174, in process_item
2023-08-04 10:15:58     result_bbox = subset.subset(**subset_params)
2023-08-04 10:15:58   File "/home/dockeruser/.local/lib/python3.9/site-packages/podaac/subsetter/subset.py", line 1284, in subset
2023-08-04 10:15:58     recombine_grouped_datasets(datasets, output_file, start_date)
2023-08-04 10:15:58   File "/home/dockeruser/.local/lib/python3.9/site-packages/podaac/subsetter/group_handling.py", line 129, in recombine_grouped_datasets
2023-08-04 10:15:58     _rename_variables(dataset, base_dataset, start_date)
2023-08-04 10:15:58   File "/home/dockeruser/.local/lib/python3.9/site-packages/podaac/subsetter/group_handling.py", line 196, in _rename_variables
2023-08-04 10:15:58     var_group.variables[new_var_name].setncatts(var_attrs)
2023-08-04 10:15:58   File "src/netCDF4/_netCDF4.pyx", line 4618, in netCDF4._netCDF4.Variable.setncatts
2023-08-04 10:15:58   File "src/netCDF4/_netCDF4.pyx", line 1757, in netCDF4._netCDF4._set_att
2023-08-04 10:15:58   File "src/netCDF4/_netCDF4.pyx", line 2029, in netCDF4._netCDF4._ensure_nc_success
2023-08-04 10:15:58 AttributeError: NetCDF: Name contains illegal characters
nlenssen2013 commented 1 year ago

heres the error i get when i remove the replace '/' code, are you able to subset your collection when you maintain the '/' ?


2023-08-04 10:15:58 2023-08-04T17:15:58.596Z [debug]: ERROR:root:NetCDF: Name contains illegal characters
2023-08-04 10:15:58 Traceback (most recent call last):
2023-08-04 10:15:58   File "/home/dockeruser/.local/lib/python3.9/site-packages/harmony/cli.py", line 215, in _invoke
2023-08-04 10:15:58     (out_message, out_catalog) = adapter.invoke()
2023-08-04 10:15:58   File "/home/dockeruser/.local/lib/python3.9/site-packages/harmony/adapter.py", line 116, in invoke
2023-08-04 10:15:58     return (self.message, self._process_catalog_recursive(self.catalog))
2023-08-04 10:15:58   File "/home/dockeruser/.local/lib/python3.9/site-packages/harmony/adapter.py", line 181, in _process_catalog_recursive
2023-08-04 10:15:58     output_item = self.process_item(item.clone(), source)
2023-08-04 10:15:58   File "/home/dockeruser/.local/lib/python3.9/site-packages/podaac/subsetter/subset_harmony.py", line 174, in process_item
2023-08-04 10:15:58     result_bbox = subset.subset(**subset_params)
2023-08-04 10:15:58   File "/home/dockeruser/.local/lib/python3.9/site-packages/podaac/subsetter/subset.py", line 1284, in subset
2023-08-04 10:15:58     recombine_grouped_datasets(datasets, output_file, start_date)
2023-08-04 10:15:58   File "/home/dockeruser/.local/lib/python3.9/site-packages/podaac/subsetter/group_handling.py", line 129, in recombine_grouped_datasets
2023-08-04 10:15:58     _rename_variables(dataset, base_dataset, start_date)
2023-08-04 10:15:58   File "/home/dockeruser/.local/lib/python3.9/site-packages/podaac/subsetter/group_handling.py", line 196, in _rename_variables
2023-08-04 10:15:58     var_group.variables[new_var_name].setncatts(var_attrs)
2023-08-04 10:15:58   File "src/netCDF4/_netCDF4.pyx", line 4618, in netCDF4._netCDF4.Variable.setncatts
2023-08-04 10:15:58   File "src/netCDF4/_netCDF4.pyx", line 1757, in netCDF4._netCDF4._set_att
2023-08-04 10:15:58   File "src/netCDF4/_netCDF4.pyx", line 2029, in netCDF4._netCDF4._ensure_nc_success
2023-08-04 10:15:58 AttributeError: NetCDF: Name contains illegal characters

Yes, all our collections that have been onboarded have slashes in the attributes