Problem assigning ACDC

[paorozo]

Hi guys, I am having a problem during the assignment of two ACDCs. E.g. prozober_ACDC0_task_HIG-RunIISummer15GS-02578__v1_T_170322_130824_743, I am using https://github.com/CMSCompOps/WmAgentScripts/blob/master/assign.py for the assignment.

The script: python assign.py -s T2_UK_SGrid_Bristol,T2_UK_London_Brunel --xrootd -w prozober_ACDC0_task_HIG-RunIISummer15GS-02578__v1_T_170322_130824_743

sends to reqmgr the dictionary [*], but I got this error:

400 Bad Request
<p>Invalid spec parameter value: Validation failed: Team parameter is mandatory. Definition: {'attr': 'team', 'default': '', 'assign_optional': False, 'optional': True, 'validate': None, 'null': False, 'type': &lt;function safeStr at 0x7f8212f3bd70&gt;}</p>

The team parameter is set in the dictionary, what could be wrong? Thanks!

[*]

{
  "SoftTimeout": 159600, 
  "BlockCloseMaxEvents": 2000000, 
  "MaxMergeEvents": 200000, 
  "CustodialSites": [], 
  "MaxRSS": 6041600, 
  "ProcessingString": {
    "HIG-RunIISummer16MiniAODv2-02961_0": "PUMoriond17_80X_mcRun2_asymptotic_2016_TrancheIV_v6", 
    "HIG-RunIISummer16DR80Premix-02945_0": "PUMoriond17_80X_mcRun2_asymptotic_2016_TrancheIV_v6", 
    "HIG-RunIISummer15GS-02578_0": "MCRUN2_71_V1", 
    "HIG-RunIISummer16DR80Premix-02945_1": "PUMoriond17_80X_mcRun2_asymptotic_2016_TrancheIV_v6"
  }, 
  "checkboxprozober_ACDC0_task_HIG-RunIISummer15GS-02578__v1_T_170322_130824_743": "checked", 
  "TrustSitelists": true, 
  "UnmergedLFNBase": "/store/unmerged", 
  "CustodialSubType": "Replica", 
  "TrustPUSitelists": true, 
  "GracePeriod": 300, 
  "MergedLFNBase": "/store/mc", 
  "SiteWhitelist": [
    "T2_UK_SGrid_Bristol", 
    "T2_UK_London_Brunel"
  ], 
  "AcquisitionEra": {
    "HIG-RunIISummer16MiniAODv2-02961_0": "RunIISummer16MiniAODv2", 
    "HIG-RunIISummer16DR80Premix-02945_0": "RunIISummer16DR80Premix", 
    "HIG-RunIISummer15GS-02578_0": "RunIISummer15GS", 
    "HIG-RunIISummer16DR80Premix-02945_1": "RunIISummer16DR80Premix"
  }, 
  "NonCustodialSubType": "Replica", 
  "NonCustodialSites": [], 
  "MinMergeSize": 2147483648, 
  "BlockCloseMaxWaitTime": 172800, 
  "MaxMergeSize": 4294967296, 
  "NonCustodialGroup": "DataOps", 
  "ProcessingVersion": 1, 
  "Dashboard": "reprocessing", 
  "Team": "production", 
  "AutoApproveSubscriptionSites": [], 
  "MaxVSize": 4394967000, 
  "SiteBlacklist": [], 
  "action": "Assign", 
  "Teamproduction": "checked", 
  "RequestStatus": "assigned", 
  "RequestName": "prozober_ACDC0_task_HIG-RunIISummer15GS-02578__v1_T_170322_130824_743"
}

[ticoann]

Hmm that is weird. Could you print out the dictionary right before the put call is made. right above the below call. (print schema)

https://github.com/CMSCompOps/WmAgentScripts/blob/master/reqMgrClient.py#L1252

[amaltaro]

I didn't follow the whole code, but there seems to have a flaw in that function reqmgr1_to_2_Assignment. BTW, since all these requests are ReqMgr2 now, why would you translate reqmgr1 "stuff" to reqmgr2?

[paorozo]

Thanks Seangchan, here it is

{'SoftTimeout': 159600, 'BlockCloseMaxEvents': 2000000, 'MaxMergeEvents': 200000, 'CustodialSites': [], 'MaxRSS': 6041600, 'ProcessingString': {u'HIG-RunIISummer16MiniAODv2-02961_0': u'PUMoriond17_80X_mcRun2_asymptotic_2016_TrancheIV_v6', u'HIG-RunIISummer16DR80Premix-02945_0': u'PUMoriond17_80X_mcRun2_asymptotic_2016_TrancheIV_v6', u'HIG-RunIISummer15GS-02578_0': u'MCRUN2_71_V1', u'HIG-RunIISummer16DR80Premix-02945_1': u'PUMoriond17_80X_mcRun2_asymptotic_2016_TrancheIV_v6'}, 'TrustSitelists': True, 'UnmergedLFNBase': '/store/unmerged', 'CustodialSubType': 'Replica', 'TrustPUSitelists': True, 'GracePeriod': 300, 'MergedLFNBase': u'/store/mc', 'SiteWhitelist': ['T2_UK_SGrid_Bristol', 'T2_UK_London_Brunel'], 'AcquisitionEra': {u'HIG-RunIISummer16MiniAODv2-02961_0': u'RunIISummer16MiniAODv2', u'HIG-RunIISummer16DR80Premix-02945_0': u'RunIISummer16DR80Premix', u'HIG-RunIISummer15GS-02578_0': u'RunIISummer15GS', u'HIG-RunIISummer16DR80Premix-02945_1': u'RunIISummer16DR80Premix'}, 'NonCustodialSubType': 'Replica', 'NonCustodialSites': [], 'MinMergeSize': 2147483648, 'BlockCloseMaxWaitTime': 172800, 'MaxMergeSize': 4294967296, 'NonCustodialGroup': 'DataOps', 'ProcessingVersion': 1, 'Dashboard': 'reprocessing', 'AutoApproveSubscriptionSites': [], 'MaxVSize': 4394967000, 'SiteBlacklist': [], 'Teamproduction': 'checked', 'RequestStatus': 'assigned', 'RequestName': 'prozober_ACDC0_task_HIG-RunIISummer15GS-02578__v1_T_170322_130824_743'}

[ticoann]

@prozober, Hi Paola, Above dictionary doesn't contain "Team", It must be removed between the first dictionary and the second one somewhere in the code.

[paorozo]

That's weird, I tried to assign 140 ACDCs and only 2 had failures in the assignment. prozober_ACDC0_task_HIG-RunIISummer15GS-02578__v1_T_170322_130824_743 prozober_ACDC0_task_HIG-RunIISummer15GS-02444__v1_T_170322_130359_9454

[paorozo]

Hmmm well, I assigned the ACDC https://cmsweb.cern.ch/reqmgr2/fetch?rid=prozober_ACDC0_task_HIG-RunIISummer15GS-02604__v1_T_170322_171402_875 and the dictionary sent was [**]. "Team" is there, what is different?

[**] {'SoftTimeout': 159600, 'BlockCloseMaxEvents': 2000000, 'MaxMergeEvents': 200000, 'CustodialSites': [], 'MaxRSS': 6041600, 'ProcessingString': {u'HIG-RunIISummer15GS-02604_0': u'MCRUN2_71_V1', u'HIG-RunIISummer16MiniAODv2-03013_0': u'PUMoriond17_80X_mcRun2_asymptotic_2016_TrancheIV_v6', u'HIG-RunIISummer16DR80Premix-02997_1': u'PUMoriond17_80X_mcRun2_asymptotic_2016_TrancheIV_v6', u'HIG-RunIISummer16DR80Premix-02997_0': u'PUMoriond17_80X_mcRun2_asymptotic_2016_TrancheIV_v6'}, 'TrustSitelists': True, 'UnmergedLFNBase': '/store/unmerged', 'CustodialSubType': 'Replica', 'TrustPUSitelists': True, 'GracePeriod': 300, 'MergedLFNBase': u'/store/mc', 'SiteWhitelist': ['T2_UK_SGrid_Bristol', 'T2_UK_London_Brunel', 'T2_UK_London_IC'], 'AcquisitionEra': {u'HIG-RunIISummer15GS-02604_0': u'RunIISummer15GS', u'HIG-RunIISummer16MiniAODv2-03013_0': u'RunIISummer16MiniAODv2', u'HIG-RunIISummer16DR80Premix-02997_1': u'RunIISummer16DR80Premix', u'HIG-RunIISummer16DR80Premix-02997_0': u'RunIISummer16DR80Premix'}, 'NonCustodialSubType': 'Replica', 'NonCustodialSites': [], 'MinMergeSize': 2147483648, 'BlockCloseMaxWaitTime': 172800, 'MaxMergeSize': 4294967296, 'NonCustodialGroup': 'DataOps', 'ProcessingVersion': 1, 'Dashboard': 'reprocessing', 'Team': 'production', 'AutoApproveSubscriptionSites': [], 'MaxVSize': 4394967000, 'SiteBlacklist': [], 'RequestStatus': 'assigned', 'RequestName': 'prozober_ACDC0_task_HIG-RunIISummer15GS-02604__v1_T_170322_171402_875'}

[vlimant]

the issue is actually in reqmgr1_to_2_Assignment that did not like a partial migration to reqmgr2 only assignment. https://github.com/CMSCompOps/WmAgentScripts/commit/2e28a572df7daafe0777e20ecf251368c289aa3f

Now assign.py will do assignment with reqmgr2 parameters all the time https://github.com/CMSCompOps/WmAgentScripts/commit/78e428bece6368975e996c6b69cfb2e761291cce

[ticoann]

Thanks Jean-Roch