search

aodn / python-aodncore

The starting point for any new pipeline handler development containing all of the core functionality of each handler
https://aodn.github.io/python-aodncore/
GNU General Public License v3.0
1 stars 1 forks source link

Compliance checker output no longer appears in pipeline log #233

Open mhidas opened 3 years ago

mhidas commented 3 years ago

Originally, when a file fails the compliance checks, the output from the checker detailing issues with the file used to be included in the pipeline logs, like this:

2020-05-04 11:24:30,000 ERROR tasks.ANMN_NRS_BGC[c50a3498-1663-4a40-8c76-a2d667aba5d0] log for failed file 'IMOS_ANMN-NRS_CDEKOSTUZ_20190118T005755Z_NRSNSI_FV01_Profile-SBE19plus_C-20190916T235239Z.nc'

------------------------------------------------------------------------------------------------------------------------
                                             IOOS Compliance Checker Report                                             
                                                        imos:1.4                                                        
                   http://content.aodn.org.au/Documents/IMOS/Conventions/IMOS_NetCDF_Conventions.pdf                    
------------------------------------------------------------------------------------------------------------------------
                                                   Corrective Actions                                                   
IMOS_ANMN-NRS_CDEKOSTUZ_20190118T005755Z_NRSNSI_FV01_Profile-SBE19plus_C-20190916T235239Z.nc has 1 potential issue

                                                        Required                                                        
------------------------------------------------------------------------------------------------------------------------
Name       Reasoning
abstract:  Global attribute 'abstract' missing.
2020-05-04 11:24:30,000 ERROR tasks.ANMN_NRS_BGC[c50a3498-1663-4a40-8c76-a2d667aba5d0] ComplianceCheckFailedError: the following files failed the check step: ['IMOS_ANMN-NRS_CDEKOSTUZ_20190118T005755Z_NRSNSI_FV0
0_Profile-SBE19plus_C-20190916T235239Z.nc', 'IMOS_ANMN-NRS_CDEKOSTUZ_20190118T005755Z_NRSNSI_FV01_Profile-SBE19plus_C-20190916T235239Z.nc']

These lines are still appearing in some pipeline logs, but I've found several cases where they are missing, e.g. tasks.ANMN_WA[1c0fc565-74fc-4d88-a5ce-5a6da8394625]. This means a file is failing the checks, but it's not easy to see what the problem is from the log.

mhidas commented 3 years ago

See tasks.DWM_SOTS[27fe365b-bc71-4d9f-97bf-e1ac3619079c] for another example