Add num_kcor_nrgf_avg_fits column in mlso_numfiles

[mgalloy]

We currently count pB FITS files into one of 3 columns:

• num_kcor_pb_fits
• num_kcor_pb_avg_fits
• num_kcor_pb_extavg_fits

But there are only 2 columns to count NRGF files:

• num_kcor_nrgf_fits
• num_kcor_nrgf_extavg_fits

So we need to add:

• num_kcor_nrgf_avg_fits

and count the NRGF FITS files correctly when adding this row into mlso_numfiles.

Tasks

• [x] add num_kcor_nrgf_avg_fits column to mlso_numfiles
• [x] count NRGF files correctly for updating mlso_numfiles
• [x] make sure "removed" files are removed from the database as well

[mgalloy]

Can add the column with:

alter table mlso_numfiles add column num_kcor_nrgf_avg_fits smallint(5) after num_kcor_nrgf_fits;

[mgalloy]

Testing on 20131113 in raw.latest in the new location...

[mgalloy]

For the 20131113 I have now:

Field	Number
num_kcor_pb_fits	234
num_kcor_pb_avg_fits	62
num_kcor_pb_extavg_fits	2
num_kcor_nrgf_fits	233
num_kcor_nrgf_avg_fits	62
num_kcor_nrgf_extavg_fits	2

[mgalloy]

The file that is in pB, but not NRGF corresponds to 20131113_193716_kcor.fts.gz, which was removed, i.e., in the removed directory/listing. This will needs to be removed from the database as well.

[mgalloy]

OK, I think this is working on the test day now.

[jburkepile]

Dear Mike:

Looks like the pipeline is reporting the number of pB_avg plus the pB_avg_enhanced. Same for the nrgf files. I think its better to report the number of pB_avg only (same for the nrgf) and not include enhanced since that doubles the number.

This is what the pipeline produced in the raw.latest run for 20131113:

• l2_pb.fts.gz : 233
• l2_pb_avg.fts.gz: 31
• l2_pb_extavg.fts.gz: 1
• l2_pb_enhanced.fts.gz: NONE (that is correct since we only produced enhanced avgs.)
• l2_pb_avg_enhanced.fts.gz: 31
• l2_pb_extavg_enhanced.fts.gz: 1

The pipeline correctly reports that the numbers for the different nrgf image types are identical to the pB.

[mgalloy]

For the 20131113 I have now:

Field	Number
num_kcor_pb_fits	233
num_kcor_pb_avg_fits	31
num_kcor_pb_extavg_fits	1
num_kcor_nrgf_fits	233
num_kcor_nrgf_avg_fits	31
num_kcor_nrgf_extavg_fits	1