search

cmsdaq / DAQExpert

New expert system processing data model produced by DAQAggregator
1 stars 2 forks source link

New instructions for GEM FED 1467 #232

Open mommsen opened 5 years ago

mommsen commented 5 years ago

There are new special instructions for GEM FED 1467. They have been added to the shift bulletin board. It would be nice if the DAQExpert could also display them:

  1. If GEM are OUT_OF_SYNC -> Perform the hard reset without calling the GEM DOC.
  2. If GEMs give events out of sequence (RU is in sync loss) -> stop/start the run, then call the GEM DOC.
  3. If GEMs require green recycle -> call GEM DOC BEFORE performing it. GEM DOC will take debug information, then proceed with the green recycle.
  4. During collisions, if GEMs stop the run more than once, they are to be immediately taken out (notify the GEM DOC after).
  5. If the GEM FM goes into ERROR with the message "...OptoHybrid connected on link X to AMC in slot Y is not responding", please call the GEM DOC, as this indicates a communication failure with the front-end and likely requires an LV power-cycle to be performed by the GEM DOC.

The last point is old and might already be implemented in the DAQExpert.

andreh12 commented 5 years ago

some comments:

andreh12 commented 5 years ago

on 1: in fact, the latest occurrence of this is http://daq-expert.cms/DAQExpert/?start=2018-08-09T20:38:12.755Z&end=2018-08-09T20:39:12.755Z and the instructions say that one should do a HardReset (up to two actually) and only call the GEM DOC if the problem is not fixed.

andreh12 commented 5 years ago

on 2: I could find only one case in the database where this happened so far:

In fact, two modules (LegacyFlowchartCase1 and OutOfSequenceData) fired but the message shown to the shifter was from LegacyFlowchartCase1 as far as I can tell.

andreh12 commented 5 years ago

item 2 is implemented in 272034a for LegacyFlowchartCase1 .

It is not clear to me what the difference is between LegacyFlowchartCase1 and OutOfSequenceData and why for the snapshot mentioned above only LegacyFlowchartCase1 but not OutOfSequenceData fires.

For OutOfSequenceData this is implemented in 81dd957 but I don't know how to test it since it does not fire the above snapshot.

andreh12 commented 5 years ago

merged the code changes for 2. into dev

andreh12 commented 5 years ago

sorry, for some reason the commits mentioned earlier were not merged into dev but I had to cherry-pick them again into a new branch feature/gem-messages-issue-232. The branch is now merged into dev. And all 212 tests passed on dev @ aafee39a .

gladky commented 5 years ago

I have requested a GL for deployment of 2.13.6. Do you want me to include this feature too? I would then make a new release 2.13.7 and update the request. Please let me know.

andreh12 commented 5 years ago

if it does not cause too much trouble, yes, please include it as it has been requested some time ago (it probably could have been a hotfix). Thanks !

mommsen commented 5 years ago

In the meantime, the short-term instructions changed again a bit: https://twiki.cern.ch/twiki/bin/viewauth/CMS/ShiftNews#Short_term_special_instructions

andreh12 commented 5 years ago

the merge commit aafee39 only addressed item 2 which corresponds to item 4 on the twiki: If GEMs give events out of sequence, run needs to be stopped/started, not necessary to call the GEM DOC beforehand. Call after run stop/start..

As you can see from the merge commit, the instructions now are:

"<<StopAndStartTheRun>>",
"Call the GEM DOC"

I'd say this is equivalent, without emphasising that the DOC should not be called before performing the action (but only afterwards).

andreh12 commented 5 years ago

Item 2 of the instructions on the twiki: While GEMs are included in the global run, if TCDS is reconfigured then please also reconfigure (i.e., green recycle) GEMs afterwards. should go into RCMS rules for the blinking items below the red/green recycle symbols.

Phil2812 commented 5 years ago

Hannes implemented this indicator in the LV0 some weeks ago: https://svnweb.cern.ch/trac/rcms/ticket/510

On Thu, Aug 23, 2018, 18:20 andreh12 notifications@github.com wrote:

Item 2 of the instructions on the twiki: While GEMs are included in the global run, if TCDS is reconfigured then please also reconfigure (i.e., green recycle) GEMs afterwards. should go into RCMS rules for the blinking items below the red/green recycle symbols.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/cmsdaq/DAQExpert/issues/232#issuecomment-415477987, or mute the thread https://github.com/notifications/unsubscribe-auth/AApiiUY8ZVVZuaxLa6eW8TF9WU9S8nx5ks5uTtZggaJpZM4Vzgwr .