Open wfastrononomer opened 1 month ago
Can't see any of the header values from updateHeader in the catalogue FITS: MAPID, CATLGID, VERSNUM, PROGID.
Has it gone to ingest before completion? MapFrameStatus - bitProcessingFlag --> complete, but why not updateHeaders: catID=3720. Check logs multiframeID=4850203 Weird - frame seems to have been created on disk >= 20240713, but only entry in logs is cuEventID=1651, 20240530.
But, /disk62/sys/test/products/map/20240713_v6/v20180110_00204_st_map160_2_1.fits does have these keys in the extension headers.
updateHeaders in filterCat bit of code --> check FITS files to see if this has been run
TODO: add more checks incase network issues - make sure updates have happened before updating bitProcessingFlag
Rerun after refactoring still gives this error. Check why.
Also pipeline should fall over, rather than continuing --> check why it doesn't - this is cu34 code.
Mostly updated the pipeline. Running to test it. Mostly working. Think last bug was a network issue, but doing more checks to be sure.
14/AUG/25 @wfastrononomer reported
Traceback (most recent call last): File "/unas/njc/VDFS/ListDrivenClean/src/curation/invocations/cu4/cu4.py", line 483, in _onRun histories) File "/unas/njc/VDFS/ListDrivenClean/src/curation/wsatools/File.py", line 297, in pickleWrite ofh = open(self.name, 'w')
2024-07-17 12:18:47.46: [14071] Getting all programme IDs.
2024-07-17 12:18:47.65: str: [14071] ValueError('Map /disk62/sys/test/products/map/20240713_v6/v20140224_00217_st_map160_2_1.fits should have known programmeID: probably a missing PROGID key',)
Traceback (most recent call last): File "/unas/njc/VDFS/ListDrivenClean/src/curation/invocations/cu4/cu4.py", line 370, in _onRun catsForProg = self.getProgIDs() File "/unas/njc/VDFS/ListDrivenClean/src/curation/invocations/cu4/cu4.py", line 745, in getProgIDs self.getProgramIDs(self._shortListFile.name) File "/unas/njc/VDFS/ListDrivenClean/src/curation/wsatools/DbConnect/IngCuSession.py", line 988, in getProgramIDs "probably a missing PROGID key" % fileName)
2024-07-17 12:18:47.65: [14071] Log written to /disk78/genOps/vsa/logs/TestVSAnjcUVDR6/cu34id14071.log