cmsdaq / DAQExpert

New expert system processing data model produced by DAQAggregator
1 stars 2 forks source link

Deadtime-analysis vs Dataflow-stuck-analysis #165

Open gladky opened 6 years ago

gladky commented 6 years ago

There are some test cases where both deadtime analysis and dataflow stuck analysis yield results at the same time. There are 3 possibilities proposed:

  1. We run deadtime analysis even when there is no rate
  2. We run deadtime analysis only when there is some rate
  3. We run deadtime analysis also for no rate but we suppress the results in case dataflow stuck found problem

Let's go case but case to decide which approach we follow.

Case 2:

http://daq-expert.cms/daq2view-react/index.html?setup=cdaq&time=2017-09-14-15:07:10

Expected results:

HLT problem: All BUs are in Blocked or CloudThis causes backpressure at FED 636 in partition EB+ of ECAL

Results to confirm:

Backpressure from HLT: DAQ backpressure coming from Filter Farm. EVM has few (0 requests, the threshold is <100) requests. Large fraction (100%, the threshold is >30%) of BUs not enabled

Case 6:

http://daq-expert.cms/daq2view-react/index.html?setup=cdaq&time=2017-06-09-17:21:55

Expected results:

Out of sequence data received: Run blocked by out-of-sync data from FED {{PROBLEM-FED}}, RU {{RU}} is in syncloss. Problem FED belongs to TTCP {{PROBLEM-TTCP}} in {{PROBLEM-SUBSYSTEM}} subsystem. Original error message: {{ORIGERRMSG}}

Results to confirm:

Backpressure from Event Builder: Backpressure from Event Building (i.e. not from HLT)

mommsen commented 6 years ago

W/o looking at all cases, I would say that dataflow stuck and deadtime analysis should be mutually exclusive. I.e. if the rate is 0, it does not make sense to run the deadtime analysis, imho.

Remi