ORAC-CC / orac

Optimal Retrieval of Aerosol and Cloud
GNU General Public License v3.0
28 stars 19 forks source link

Update ORAC main processor to enable customised filenames for output #69

Closed simonrp84 closed 1 year ago

simonrp84 commented 2 years ago

This is primarily a kicking-off point for discussion on how to incorprorate custom file naming into the ORAC main processor. The issue we have is that currently the filename is decided from the input filename (from the preprocessor) and the LUT class. With the new ORAC LUTs we now have multiple LUT types for the same class (for example, various ice habits) and each of these would produce the same output filename - which is undesirable. A workaround is to save each LUT output to its own directory but that seems very messy. Here, I add a variable to the main processor that allows the user to set a 'comment' in their driver file that then gets used in the ORAC filenames. For example, when processing the general habit mixture for ICE I can set: Ctrl%FID%L2_comment = AGG and the output filename will end with _AGG.primary.nc.

This gets around the issue but might not be the best solution. Let's discuss what we should do. Options I can think of. 1) Keep the status quo. Works with all our existing code but presents problems with multiple LUTs. 2) Add an attribute inside each LUT that adds a LUT specific 'comment' to the filename. This is less flexible but also less likely to break other parts of the code. 3) This approach. Very flexible, but could break the python scripts and post-processor.

adamcpovey commented 1 year ago

What do you think of how Andy dealt with this?

I'd personally prefer ORAC filenames to be deducible from the L1 file and the LUT alone, as it makes it easier to write "did I process everything" scripts.

simonrp84 commented 1 year ago

Sorry, how did Andy deal with this?

adamcpovey commented 1 year ago

https://github.com/ORAC-CC/orac/blob/c8a841198802cf7a14960c02fc6394a8c580b3da/src/read_driver.F90#L453-L454

and

https://github.com/ORAC-CC/orac/blob/c8a841198802cf7a14960c02fc6394a8c580b3da/tools/pyorac/definitions.py#L585-L591

He uses the full name of the file, with various identifying numbers.