wfastrononomer
commented
6 days ago Rearranged code to set fileCompsOfProd at the beginning
Open wfastrononomer opened 6 days ago
wfastrononomer
commented
6 days ago Rearranged code to set fileCompsOfProd at the beginning
" # 2024-09-13 12:22:07.27: Programming Error: KeyError: 468
Traceback (most recent call last): File "/huni/scos/VDFS/trunk/src/curation/wsatools/DbConnect/CuSession.py", lin e 479, in run self._onRun() File "./cu13.py", line 631, in _onRun self.createProducts(newProducts, productOfID, componentsOfProd) File "./cu13.py", line 1091, in createProducts self.createTiles(newProducts, productOfID, componentsOfProd) File "./cu13.py", line 1857, in createTiles productOfID, componentsOfProd, filtCompsOfProd) File "./cu13.py", line 2008, in prllProcMosaic ','.join([comp.imageName for comp in filtCompsOfProd[productID]])), alwaysLo g=False)
First check ProductLinks to find which deep stack products produce this tile.
" select intProductID from ProductLinks where programmeID=170 and combiProductID=468
| | intProductID | |
|>| 2754 |<| |>| 2793 |<| |>| 2813 |<| |>| 2866 |<| |>| 2885 |<| |>| 2945 |<|
Check that the processing of them is complete and find their names: bitProcessingFlag=2^30 is complete.
" select productID,fileName,bitProcessingFlag from ProductProcessing where programmeID=170 and productType='stack' and productID in (select intProductID from ProductLinks where programmeID=170 and combiProductID=468)
| | productID | fileName | bitProcessingFlag | |
|>| 2754 | /disk67/vsa/products/stacks/20240809_v2/e20240809_00170002754_dp_st.fit | 1073741824 |<| |>| 2793 | /disk67/vsa/products/stacks/20240809_v2/e20240809_00170002793_dp_st.fit | 1073741824 |<| |>| 2813 | /disk67/vsa/products/stacks/20240809_v2/e20240809_00170002813_dp_st.fit | 1073741824 |<| |>| 2866 | /disk67/vsa/products/stacks/20240809_v2/e20240809_00170002866_dp_st.fit | 1073741824 |<| |>| 2885 | /disk67/vsa/products/stacks/20240809_v2/e20240809_00170002885_dp_st.fit | 1073741824 |<| |>| 2945 | /disk67/vsa/products/stacks/20240809_v2/e20240809_00170002945_dp_st.fit | 1073741824 |<|
All good. They exist. Now check the processing status of the tile:
" select productID,filename,bitProcessingFlag from ProductProcessing where programmeID=170 and productType='tile' and productID=468
| | productID | filename | bitProcessingFlag | |
|>| 468 | /disk67/vsa/products/tiles/20240906_v2/e20240906_00170000468_dp_st_tl.fit | 24 |<|
24 = 2^3 +2^4 --> filtered stacks, and created unfiltered tile [see lines 118 onwards of cu13.py].
Check what happened with filtering:
grep -B 5 -A 20 "Filtering" /disk78/genOps/vsa/logs/VSAVVV15_cu13id504182.log | more
" # 2024-09-13 12:22:07.17: Filtering... " # 2024-09-13 12:22:07.20: Mosaicing unfiltered pawprints ... " # 2024-09-13 12:22:07.24: Mosaicing filtered pawprints ... " # 2024-09-13 12:22:07.27: Programming Error: KeyError: 468
Filtering and mosaicing happened very quickly - this suggests these had already been completed on a previous run.
All expected filtered stacks do exist though: ls /disk67/vsa/products/tiles/20240906_v2/tmp_e20240809_00170002754_dp_st.fit
Not sure why filtCompsOfProd[468] doesn't exist?
Check cu13 code to see where filtCompsOfProd is set up.