Strange timing for the "archive build reports" job

[hpages]

I noticed that the crontab entries for archiving the last 6 reports were located at the bottom of the crontab. Since these entries are about archiving the reports for the software builds, I took the liberty to move them up to the BIOC 3.19 SOFTWARE BUILDS section. Sounds like a more natural place.

However, after doing so, I realized that the archiving happens 10 min before the start of the postrun.sh script. Is that intended? Here is the full BIOC 3.19 SOFTWARE BUILDS section:

# BIOC 3.19 SOFTWARE BUILDS
# -------------------------

# prerun:
00 14 * * 0-5 /bin/bash --login -c 'cd /home/biocbuild/BBS/3.19/bioc/`hostname` && ./prerun.sh >>/home/biocbuild/bbs-3.19-bioc/log/`hostname`-`date +\%Y\%m\%d`-prerun.log 2>&1'

# run:
00 15 * * 0-5 /bin/bash --login -c 'cd /home/biocbuild/BBS/3.19/bioc/`hostname` && ./run.sh >>/home/biocbuild/bbs-3.19-bioc/log/`hostname`-`date +\%Y\%m\%d`-run.log 2>&1'

# fetch kunpeng2 results (every hour):
25 15-23 * * 0-5 /usr/bin/rsync -azv 114.119.187.96::bioc /home/biocbuild/public_html/BBS/3.19/bioc/products-in/kunpeng2 >>/home/biocbuild/bbs-3.19-bioc/log/fetch_kunpeng2_results.log 2>&1

## NEXT DAY

# keep fetching kunpeng2 results (every hour):
25 00-11 * * 1-6 /usr/bin/rsync -azv 114.119.187.96::bioc /home/biocbuild/public_html/BBS/3.19/bioc/products-in/kunpeng2 >>/home/biocbuild/bbs-3.19-bioc/log/fetch_kunpeng2_results.log 2>&1

# postrun (make sure this starts AFTER 'biocbuild' has finished its "run.sh" job on ALL the nodes):
45 12 * * 1-6 /bin/bash --login -c 'cd /home/biocbuild/BBS/3.19/bioc/`hostname` && ./postrun.sh >>/home/biocbuild/bbs-3.19-bioc/log/`hostname`-`date +\%Y\%m\%d`-postrun.log 2>&1'

# archive build reports:
35 12 * * 1-6 cp /home/biocbuild/public_html/BBS/3.19/bioc/report/report.tgz /home/biocbuild/archives/bioc-report-`date +\%Y\%m\%d`.tgz

# remove reports older than 1 week
40 12 * * 1-6 find /home/biocbuild/archives/bioc-report*.tgz -maxdepth 1 -mtime +7 -type f -delete

# automatic build failure notifications (sent MWF):
#00 13 * * 1 cd /home/biocbuild/BBS/3.19/bioc/`hostname` && ./stage7-notify.sh >>/home/biocbuild/bbs-3.19-bioc/log/`hostname`-`date +\%Y\%m\%d`-notify.log 2>&1

I didn't check the crontab on nebbiolo2 but maybe it needs the same kind of adjustments.

H.

[hpages]

Just checked nebbiolo2 and postrun.sh is running at 11:30 am there so it was probably also running at 11:30 am on nebbiolo1 until it got changed recently (I vaguely remember you mentioned something like this on Slack recently).

So the archiving of the reports and removal of the old ones is probably meant to run after that, which makes sense.

For clarity of the crontab, these 2 jobs should probably be scheduled to happen after the "automatic build failure notifications" (and thus the entries should be moved below that).

Also, the "automatic build failure notifications" job should run more than 15 min after the postrun job, e.g. 30 or 45 min after it. It takes about 15 min for postrun to complete and this time increases slowly as the nb of packages to build grows. I would recommend that you make the change now even if the entry is commented out.

The idea of all this is to keep the crontab organized and easier to grasp, to reduce the risk of scheduling mistakes or conflicts.

Thanks

[jwokaty]

Thanks for your comments and suggestions. I've moved the archive jobs up for nebbiolo2 and spaced them 15 minutes after notifications, which are 30 minutes after postrun. I also added a note about the spacing for respective run times.