search

wfau / ScienceArchives

0 stars 0 forks source link

Check cu13 run that fell over. What went wrong, what needs cleaning up. #611

Open wfastrononomer opened 3 months ago

wfastrononomer commented 3 months ago

Hi Nick,

CU13 fell over: /disk78/genOps/vsa/logs/VSAVVV15_cu13id503942.log with loads of File "./cu13.py", line 3628, in stackImages self._target(*self._args, *self._kwargs) File "./cu13.py", line 3628, in stackImages self.run() self._target(self._args, **self._kwargs) File "/huni/scos/usr/local/lib/python2.7/multiprocessing/process.py", line 114, in run

and

IOError: [Errno 4] Interrupted system call cmdOutput = ''.join(extp.out(cmd))

It may be that this is because I started another one on ibi as well (this one was on huni) or because the number of processed files is too big. The amount of 200 seems to work better.

One other thing I noticed is that, independent of the given range it always creates all of the modified conf maps. (And it would be really helpful if it does those in an ordered fashion to see which is the first and which is the last.)

wfastrononomer commented 3 months ago

Check logs, check ProductProcessing. Find files that have been partially stacked and delete. What went wrong?

wfastrononomer commented 3 months ago

tiles 201 - 1000

select bitProcessingFlag,COUNT(*) from ProductProcessing as pp,ProductLinks as pl where pl.combiProductID>=201 and pl.combiProductID<=1000 and pp.programmeID=170 and pl.programmeID=pp.programmeID and pp.productID=pl.intProductID and pp.productType='stack' and pl.intProdType=pp.productType group by pp.bitProcessingFlag

| | bitProcessingFlag | | |

|>| 0 | 38 |<| |>| 1 | 1976 |<|

There are 38, which were in the process of being stacked:

select fileName from ProductProcessing as pp,ProductLinks as pl where pl.combiProductID>=201 and pl.combiProductID<=1000 and pp.programmeID=170 and pl.programmeID=pp.programmeID and pp.productID=pl.intProductID and pp.productType='stack' and pl.intProdType=pp.productType and pp.bitProcessingFlag=0

| | fileName | |

|>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003457_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003459_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003465_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003453_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003467_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003480_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003494_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003481_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003495_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003482_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003496_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003483_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003497_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003484_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003498_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003454_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003485_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003499_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003456_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003487_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003501_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003471_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003477_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003473_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003479_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003460_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003461_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003462_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003464_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003468_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003470_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003488_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003491_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003490_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003493_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003474_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003476_dp_st.fit |<| |>| /disk67/vsa/products/stacks/20240809_v2/e20240809_00170003502_dp_st.fit |<|