WP3.5.3 D Conduct full SXDS VIDEO pipeline run with HSC PDR3 and latest obs_vista version in generation 3 Butler

lsst-uk / lsst-ir-fusion

Development of code to exploit LSST and VISTA imaging

Apache License 2.0

2 stars 0 forks source link

Closed tms-epcc closed 2 years ago

tms-epcc commented 2 years ago

17/JUN/22

tms-epcc commented 2 years ago

01/JUL/22

run in progress, in mid-run, now doing the final stage
CPU time though is a concern. Using 100K core hours this time as opposed to 40K hours with gen2. DESC are responsible for development of Parsl and so have discussed possible reasons for this with them. Others using this Parsl plug-in, have reported the same issue. The Parsl developers have checked Raphael's config files , etc to confirm it is not a set-up issue. Nothing was found so agreed with the Parsl developers that this one for them to resolve rather than a user one.

tms-epcc commented 2 years ago

15/JUL/22 @raphaelshirley reported

there was a bug that led to this large run hanging and consuming CPU unnecessarily
This bug in parsl has been fixed and whilst is behaving better it is still crashing
Paul Price has written the general package for the lsst pipelines - https://github.com/lsst/ctrl_bps_parsl
Plan to test with this package.

tms-epcc commented 2 years ago

29/JUL/22

bug in calibration catalogue has been fixed.
subsequent pipeline rerun is almost finished. Hope it will be completed by 01/AUG.
this is the final run and so expect to close this ticket(task) at that point.
The tables, butler data and catalogue data from the final run will be sent to @MikeeRead (see #50).

tms-epcc commented 2 years ago

12/AUG/22

there are some minor issues but there will be more runs in the future so this can be closed
@raphaelshirley will supply link to location of data as evidence that this can be closed.

tms-epcc commented 2 years ago

26/AUG/22 @raphaelshirley reported

the directory on CSD3 is at /home/ir-shir1/rds/rds-iris-ip005/ras81/lsst-ir-fusion/dmu4/data on the csd3 space
all this will be moved to the RSP when development of that is finished.