search

cms-sw / cmssw

CMS Offline Software
http://cms-sw.github.io/
Apache License 2.0
1.07k stars 4.28k forks source link

RootOutputModule should not prefetch data products it fast copied #13454

Open Dr15Jones opened 8 years ago

Dr15Jones commented 8 years ago

If the RootOutputModule fast copied a TBranch from the source we should not prefetch the data from that branch as well. The prefetch would cause the data to be deserialized which is unnecessary and just takes unnecessary amounts of time.

cmsbuild commented 8 years ago

A new Issue was created by @Dr15Jones (Chris Jones).

@davidlange6, @smuzaffar, @degano, @davidlt, @Dr15Jones can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

Dr15Jones commented 8 years ago

assign core

cmsbuild commented 8 years ago

New categories assigned: core

@Dr15Jones,@smuzaffar you have been requested to review this Pull request/Issue and eventually sign? Thanks

smuzaffar commented 2 years ago

@Dr15Jones , @makortel has this been implemented?

makortel commented 2 years ago

I suppose "no" (also FYI @dan131riley)

dan131riley commented 2 years ago

AIUI, it's possible that a product that was fast-cloned may still need to be deserialized, especially when there are multiple output modules. I guess we'd want to initially turn off prefetch for fast-cloned branches but then turn it back on if we are getting entries fetched on the branch.

Dr15Jones commented 2 years ago

@dan131riley given fast cloning is off if we use multiple threads in a job, this optimization may not longer be of any use.

dan131riley commented 2 years ago

@dan131riley given fast cloning is off if we use multiple threads in a job, this optimization may not longer be of any use.

@Dr15Jones where does that get done? (I see it's done for the HLT, but I couldn't find a more general switch.)

Dr15Jones commented 2 years ago

The logic is here

https://github.com/cms-sw/cmssw/blob/04e25fafc456b764f3b59c4e88b1c0f1b681e368/FWCore/Framework/src/EventProcessor.cc#L906-L908

makortel commented 2 years ago

So maybe it is not worth of even trying?