ReqMgr2 fails to load ACDC request with 500 Internal Server Error

Impact of the bug ReqMgr2

Describe the bug Apparently an issue introduced in the last release candidate 2.3.6rc6, likely from this development: https://github.com/dmwm/WMCore/issues/12037

Where ReqMgr2 crashes with 500 Internal Server Error whenever we try to load an ACDC workflow in new state, e.g.: https://cmsweb-testbed.cern.ch/reqmgr2/fetch?rid=vokovale_ACDC0_TC_Backfill_IDR_CMS_Multi_240716_125212_9384

It is very likely that other statuses are facing the same issue(e.g., rejected, etc).

After performing a status transition in the GUI and re-loading the same workflow, it also gives a 500 error.

How to reproduce it Load any ACDC workflow in new status. Or perform any workflow transition in the GUI and reload the page right after the transition is submitted.

Expected behavior ReqMgr2 GUI should not crash and allowed changes per status need to work properly.

Additional context and error message Example of the traceback:

[24/Sep/2024:01:27:39] HTTP 
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/cherrypy/_cprequest.py", line 638, in respond
    self._do_respond(path_info)
  File "/usr/local/lib/python3.8/site-packages/cherrypy/_cprequest.py", line 697, in _do_respond
    response.body = self.handler()
  File "/usr/local/lib/python3.8/site-packages/cherrypy/lib/encoding.py", line 223, in __call__
    self.body = self.oldhandler(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/cherrypy/_cpdispatch.py", line 54, in __call__
    return self.callable(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/WMCore/ReqMgr/Web/ReqMgrService.py", line 601, in fetch
    table=json2table(filteredDoc, web_ui_names(), visible_attrs, selected),
  File "/usr/local/lib/python3.8/site-packages/WMCore/ReqMgr/Web/utils.py", line 141, in json2table
    key, val = cells.pop(attr)
KeyError: 'RequestStatus'
...
[24/Sep/2024:01:27:39] reqmgr2-b77b679c6-klv7w xxx "GET /reqmgr2/fetch?rid=amaltaro_ACDC_TaskChain_LumiMask_multiRun_HG2408_Val_240924_012713_5234 HTTP/1.1" 500 Internal Server Error ...

dmwm / WMCore

ReqMgr2 fails to load ACDC request with 500 Internal Server Error #12107