DC2 Run2.2i DR2 processing at CC

[johannct]

This production used the same list of visits as the DR2 production at NERSC, and in particular uses the opsim flag to populate the DDF region with only the visits of the WFD cadence.

The workdir at CC is /sps/lssttest/dataproducts/desc/DC2/Run2.2i/v19.0.0-v1/ and the data repository is rerun/run2.2i-coadd-wfd-dr2-v1 under this workdir, plus the upstream data repositories chained as usual by the pipeline.

The parquet files are under dpdd/run2.2i-wfd-dr2. Many patches are missing in the object and metacal catalogs, due to the shallowness of DR2; several jobs actually fail in metacal for this reason (as far as I could check):

object catalog:

• 2723 : only 2,6 3,6 4,6

• 2724 : only 0,6 1,6 2,6 4,6 5,6

• 2725 : only 0,6 1,6 2,6 4,6 6,6

• 2726 : only 0,6 2,6 3,6 4,6 5,6 6,6

• 2727 : only 2,6 4,6 5,6 6,6

• 2728 : only 0,6 3,6 4,6 5,6 6,6

• 2729 : only 0,6 1,6

• 2730 : only 1,6 2,6 3,6 5,6 6,6

• 2731 : only 1,6 5,6 6,6

• 2732 : only 5,6 6,6

• 2733 : only 0,6 1,6

• 2734 : only 0,6 1,6 2,6 3,6 4,6 5,6

• 2735 : entirely missing

• 2896 : only 0,0 through 1,3

• 2909 : entirely missing

• 3074 : only 0,0 through 3,2

• 3086 : 0,0 through 1,6 and 2,6 missing

• 3256 : 5,3 through 6,6 missing

• 3268 : 0,0 through 3,6 and 4,5 4,6 missing

• 3441 : only 0,0 through 0,4

• 3442 : only the x,6 row

• 3454 : only 6,0 6,1 6,2

• 3631 : only through 2,6

• 3643 : 2,0 through 6,6

• 3825 : only through 4,6

• 3837 : 0,0 through 3,6 missing

• 4022 : only 0,0 0,1 0,2

• 4224 : only through 3,5

• 4236 : 0,0 through 1,6 and 2,1 through 2,6 missing

• 4428 : entirely missing

• 4441 : only the 6,x row

• 4636 : only 0,0 through 2,1

• 4648 : 0,x row and 1,6 missing

• 4848 : only through 5,3

• 4860 : only 4,0 through 4,4 and 5,0 through 6,6

• 5062 : 0,6 and 1,6 through 6,6 missing

• 5064 : 0,6 1,6 6,6 missing

• 5065 : 4,6 missing

• 5067 : 0,6 missing

• 5068 : 2,6 and 6,6 missing

• 5069 : 6,6 missing

• 5072 : 1,6 and 6,6 missing

• 5074 : 0,x row and 1,6 3,6 missing

[johannct]

CPU usage and plots :

• for coaddDriver (unnormalized seconds): u:3337151 g:4206948 r:9067280 i:8962551 z:6875475 y:7460611 total : 39910016
• for coaddDriver (unnormalized seconds): 39700246

[johannct]

Memory usage

• for coaddDriver
• for multibandDriver

[heather999]

@johannct It sounds like I should request transfer of this processing to NERSC. I'll add a comment to the ongoing thread on the IPP repo with Fabio.

[jchiang87]

@johannct Do you have the data that went into those plots available somewhere? I expect that having used the pipe drivers means that we can't get info at the pipe task level, but it still might be useful to look at the pipe driver numbers, especially if each entry on those histograms can be associated with specific filter-tract-patch combinations.

[johannct]

@jchiang87 what I used is extremely coarse : data_from_log.bash.txt is just running over logfiles and extract basic information. I need to clean the resulting outputs as some logs are anomalous for several reasons, and ideally all this should be run after the driver execution and after some sanity check. Then I just load the output and make some plots. I will send the notebook by email. There are monitoring outputs in each log directory, put in place by Bastien Gounon, but it only shows one process it seems, so it it not very useful. Finer grained analyses would have to partition a given log per pid..... Doable but unwieldy. On the other hand my output does provide tract/filter/patch info and the path to the log file. I'll provide the cleaned outputs as well by email

[heather999]

@johannct as noted on Slack There appears to be 4 tracts missing: 3641, 4035, 4226, 4852 I don't see them in run2.2i-coadd-wfd-dr2-v1/deepCoadd-results/merged or in the associated u and grizy areas either. Any idea what happened to those tracts?

[johannct]

this is now done.