search

bnlawrence / cfs

Rebooting cfstore, leaner and meaner
0 stars 1 forks source link

CFAhandler doing work it doesn't need to do; info probably already in cf-python field and/or data properties #6

Open bnlawrence opened 2 months ago

bnlawrence commented 2 months ago

A lot of the file and information handling in CFAhandler() is probably unnecessary given the methods available in cf-python.

E.g 

for f in fh: print(f.data.cfa_get_aggregated_data())
{'address': 'cfa_address_5', 'file': 'cfa_file_5', 'format': 'cfa_format', 'location': 'cfa_location_4'}
{'address': 'cfa_address_6', 'file': 'cfa_file_4', 'format': 'cfa_format', 'location': 'cfa_location_1'}
{'address': 'cfa_address_7', 'file': 'cfa_file_6', 'format': 'cfa_format', 'location': 'cfa_location_4'}
{'address': 'cfa_address_8', 'file': 'cfa_file_7', 'format': 'cfa_format', 'location': 'cfa_location_4'}
{'address': 'cfa_address_9', 'file': 'cfa_file_8', 'format': 'cfa_format', 'location': 'cfa_location_4'}
{'address': 'cfa_address_10', 'file': 'cfa_file_9', 'format': 'cfa_format', 'location': 'cfa_location_4'}
{'address': 'cfa_address_11', 'file': 'cfa_file_10', 'format': 'cfa_format', 'location': 'cfa_location_4'}
{'address': 'cfa_address_12', 'file': 'cfa_file_11', 'format': 'cfa_format', 'location': 'cfa_location_4'}
{'address': 'cfa_address_13', 'file': 'cfa_file_12', 'format': 'cfa_format', 'location': 'cfa_location_4'}
{'address': 'cfa_address_14', 'file': 'cfa_file_13', 'format': 'cfa_format', 'location': 'cfa_location'}
{'address': 'cfa_address_353', 'file': 'cfa_file_15', 'format': 'cfa_format', 'location': 'cfa_location_6'}
{'address': 'cfa_address_354', 'file': 'cfa_file_39', 'format': 'cfa_format', 'location': 'cfa_location_7'}
{'address': 'cfa_address_355', 'file': 'cfa_file_18', 'format': 'cfa_format', 'location': 'cfa_location_7'}
{'address': 'cfa_address_356', 'file': 'cfa_file_14', 'format': 'cfa_format', 'location': 'cfa_location_5'}
{'address': 'cfa_address_660', 'file': 'cfa_file_72', 'format': 'cfa_format', 'location': 'cfa_location_27'}
{'address': 'cfa_address_661', 'file': 'cfa_file_73', 'format': 'cfa_format', 'location': 'cfa_location_28'}
{'address': 'cfa_address_662', 'file': 'cfa_file_94', 'format': 'cfa_format', 'location': 'cfa_location_38'}
{'address': 'cfa_address_663', 'file': 'cfa_file_97', 'format': 'cfa_format', 'location': 'cfa_location_41'}

provides information we could probably directly use without the code I have now, if I did but know how to reference these items. To discuss with David when he returns.

bnlawrence commented 1 month ago

The code in CFAhandler() (found in cfs/db/cfa_tools.py) could be rewritten to use cf or cfdm functionality only. This would be preferable to the way it is done now with fragile NetCDF attribute traversal. (This code is tested in the standard test suite, though extra coverage would be nice.)