search

dmwm / WMCore

Core workflow management components for CMS.
Apache License 2.0
46 stars 107 forks source link

Work unit update has Integrity Error #8200

Open ticoann opened 7 years ago

ticoann commented 7 years ago

Need a further investigation on vocms008

IntegrityError: (IntegrityError) (1452, 'Cannot add or update a child row: a foreign key constraint fails (wmagent.wmbs_frl_workunit_assoc, CONSTRAINT wmbs_frl_workunit_assoc_ibfk_2 FOREIGN KEY (fileid, run, lumi) REFERENCES wmbs_file_runlumi_map (fileid, run, lumi) ON DELETE CA)') 'INSERT INTO wmbs_frl_workunit_assoc (workunit, firstevent, lastevent, fileid, run, lumi) VALUES (LAST_INSERT_ID(), %s, %s, %s, %s, %s)' (0, 2147483647, 14453L, 193556L, 31L)

ericvaandering commented 7 years ago

Well damn. Maybe multiple jobs trying to run on the same file, run, and lumi? I'll have to look at the design, but it seems like the step or task should be in there too. Even workflow is not enough, I imagine since in the tree workflow we'll have that situation.

ticoann commented 7 years ago

Just for the record for debugging:

The problem was there is no entries for wmbs_file_runlumi_map table. For below task. /sryu_ReRecoSkim_reqmgr2_170926_221527_1536/DataProcessing/DataProcessingMergeRECOoutput/skim_2012A_BTag/skim_2012A_BTagMergeSKIMStreamHighLumi

MariaDB [wmagent]> select * from wmbs_file_runlumi_map where fileid=14453 and run=193556 and lumi=31; Empty set (0.00 sec)

I wonder whether following function is causing problem https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/DataStructs/Mask.py#L188 called by here https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/WMBS/Subscription.py#L642

FYI here are the all the lumis for above file MariaDB [wmagent]> select * from wmbs_file_runlumi_map where fileid=14453 and run=193556;
+--------+--------+------+------------+ | fileid | run | lumi | num_events | +--------+--------+------+------------+ | 14453 | 193556 | 41 | NULL | | 14453 | 193556 | 42 | NULL | | 14453 | 193556 | 43 | NULL | | 14453 | 193556 | 44 | NULL | | 14453 | 193556 | 45 | NULL | | 14453 | 193556 | 46 | NULL | | 14453 | 193556 | 47 | NULL | | 14453 | 193556 | 48 | NULL | | 14453 | 193556 | 49 | NULL | | 14453 | 193556 | 50 | NULL | +--------+--------+------+------------+

file name for 14453 - /store/unmerged/sryu/BTag/RAW-RECO/HighLumi-ReRecoSkim_reqmgr2-v11/00000/40F0CE12-39A3-E711-A9DC-FA163E7DD2FC.root