search

cms-analysis / flashgg

20 stars 158 forks source link

Weights to the Era2017_legacy_v1_Summer20UL MicroAOD catalogs #1303

Closed emanueledimarco closed 1 year ago

emanueledimarco commented 1 year ago

Adding back the weights for the Era2017_legacy_v1_Summer20UL MicroAOD campaign. All the samples have the check step passed now.

emanueledimarco commented 1 year ago

Commit ae921a2 adds weights for RunIISummer20UL18 campaign.

youyingli commented 1 year ago

Some MC samples like DYJetsToLL_M-50_TuneCP5_13TeV-amcatnloFXFX-pythia8 and so on in Era2017_legacy_v1_Summer20UL still have no weight. Could you please check again?

On the other hand, as the following datasets in RunIISummer20UL18 are empty, could you also delete them from those JSON files?

/EGamma/phys_higgs-RunIISummer20UL18-legacyRunII-v0-Run2018A-UL2018_MiniAODv2-v1-a7c88d6f8b657df10cc9b059890284c0/USER
/EGamma/phys_higgs-RunIISummer20UL18-legacyRunII-v0-Run2018B-UL2018_MiniAODv2-v1-a7c88d6f8b657df10cc9b059890284c0/USER
/EGamma/phys_higgs-RunIISummer20UL18-legacyRunII-v0-Run2018C-UL2018_MiniAODv2-v1-a7c88d6f8b657df10cc9b059890284c0/USER
/EGamma/phys_higgs-RunIISummer20UL18-legacyRunII-v0-Run2018D-UL2018_MiniAODv2-v2-a7c88d6f8b657df10cc9b059890284c0/USER
emanueledimarco commented 1 year ago

@youyingli done both (it took a bit of time since the DYjetsToLL samples had many files.

youyingli commented 1 year ago

@emanueledimarco Sorry for the late reply. Please fix GluGluHToGG_M-125_TuneCP5_13TeV-powheg-pythia8 in RunIISummer20UL18.

BTW. Although this PR fixes bugs about missing weights, I would like to remind you please update the requested MC samples to help those UL analyzers. Thanks.

emanueledimarco commented 1 year ago

@youyingli sure, I will do. To make sure we are thinking the same: For GluGluHToGG_M-125_TuneCP5_13TeV-powheg-pythia8 sample you mean that there is a duplication (a triplication) with the same name. I'll keep only the one with 1M events that should be the latest.

For what regards the requested UL MC samples, can you post them here, just to be sure of what are the missing ones (I maybe have missed some)? Thanks

youyingli commented 1 year ago

GluGluHToGG_M-125_TuneCP5_13TeV-powheg-pythia8 seems to be signed as "bad" file. I also checked it should be healthy. Maybe you can check it again.

And yes, I list the missing MicroAODs (almost photon-lepton backgrounds) in the following. Perhaps, @rohithsaradhy and @Xingfu23, the VH leptonic AC analyzers, can tell more missing samples.

On the other hand, some MC samples belonging UL18 are accidentally put in RunIISummer20UL16postVFP campaign. Could you also move them to UL18 campaign? In particular, they are almost bkg samples related to photon-lepton bkg. Can you check if they are actually listed in the following?

2016 preVFP

"/GJet_Pt-20to40_DoubleEMEnriched_MGG-80toInf_TuneCP5_13TeV_Pythia8/RunIISummer20UL16MiniAODAPVv2-106X_mcRun2_asymptotic_preVFP_v11-v4/MINIAODSIM",
"/GJet_Pt-20toInf_DoubleEMEnriched_MGG-40to80_TuneCP5_13TeV_Pythia8/RunIISummer20UL16MiniAODAPVv2-106X_mcRun2_asymptotic_preVFP_v11-v6/MINIAODSIM",
"/GJet_Pt-40toInf_DoubleEMEnriched_MGG-80toInf_TuneCP5_13TeV_Pythia8/RunIISummer20UL16MiniAODAPV-106X_mcRun2_asymptotic_preVFP_v8-v3/MINIAODSIM",
"/ZGTo2NuG_TuneCP5_13TeV-amcatnloFXFX-pythia8/RunIISummer20UL16MiniAODAPVv2-106X_mcRun2_asymptotic_preVFP_v11-v2/MINIAODSIM",
"/ZGToLLG_01J_5f_TuneCP5_13TeV-amcatnloFXFX-pythia8/RunIISummer20UL16MiniAODAPVv2-106X_mcRun2_asymptotic_preVFP_v11-v1/MINIAODSIM",
"/WGToLNuG_01J_5f_TuneCP5_13TeV-amcatnloFXFX-pythia8/RunIISummer20UL16MiniAODAPVv2-106X_mcRun2_asymptotic_preVFP_v11-v1/MINIAODSIM",
"/WW_TuneCP5_13TeV-pythia8/RunIISummer20UL16MiniAODAPVv2-106X_mcRun2_asymptotic_preVFP_v11-v1/MINIAODSIM",
"/WZ_TuneCP5_13TeV-pythia8/RunIISummer20UL16MiniAODAPVv2-106X_mcRun2_asymptotic_preVFP_v11-v1/MINIAODSIM",
"/ZZ_TuneCP5_13TeV-pythia8/RunIISummer20UL16MiniAODAPVv2-106X_mcRun2_asymptotic_preVFP_v11-v1/MINIAODSIM",
"/TTGG_TuneCP5_13TeV-amcatnlo-pythia8/RunIISummer20UL16MiniAODAPVv2-106X_mcRun2_asymptotic_preVFP_v11-v2/MINIAODSIM",
"/TTGJets_TuneCP5_13TeV-amcatnloFXFX-madspin-pythia8/RunIISummer20UL16MiniAODAPVv2-106X_mcRun2_asymptotic_preVFP_v11-v2/MINIAODSIM",
"/TGJets_TuneCP5_13TeV-amcatnlo-madspin-pythia8/RunIISummer20UL16MiniAODAPVv2-106X_mcRun2_asymptotic_preVFP_v11_ext1-v2/MINIAODSIM",
"/TTJets_TuneCP5_13TeV-amcatnloFXFX-pythia8/RunIISummer20UL16MiniAODAPVv2-106X_mcRun2_asymptotic_preVFP_v11-v1/MINIAODSIM"

2016 post

"/GJet_Pt-20to40_DoubleEMEnriched_MGG-80toInf_TuneCP5_13TeV_Pythia8/RunIISummer20UL16MiniAODv2-106X_mcRun2_asymptotic_v17-v6/MINIAODSIM",
"/GJet_Pt-20toInf_DoubleEMEnriched_MGG-40to80_TuneCP5_13TeV_Pythia8/RunIISummer20UL16MiniAODv2-106X_mcRun2_asymptotic_v17-v4/MINIAODSIM",
"/GJet_Pt-40toInf_DoubleEMEnriched_MGG-80toInf_TuneCP5_13TeV_Pythia8/RunIISummer20UL16MiniAOD-106X_mcRun2_asymptotic_v13-v3/MINIAODSIM",
"/ZGTo2NuG_TuneCP5_13TeV-amcatnloFXFX-pythia8/RunIISummer20UL16MiniAODv2-106X_mcRun2_asymptotic_v17-v2/MINIAODSIM",
"/ZGToLLG_01J_5f_TuneCP5_13TeV-amcatnloFXFX-pythia8/RunIISummer20UL16MiniAODv2-106X_mcRun2_asymptotic_v17-v1/MINIAODSIM",
"/WGToLNuG_01J_5f_TuneCP5_13TeV-amcatnloFXFX-pythia8/RunIISummer20UL16MiniAODv2-106X_mcRun2_asymptotic_v17-v1/MINIAODSIM",
"/WW_TuneCP5_13TeV-pythia8/RunIISummer20UL16MiniAODv2-106X_mcRun2_asymptotic_v17-v1/MINIAODSIM",
"/WZ_TuneCP5_13TeV-pythia8/RunIISummer20UL16MiniAODv2-106X_mcRun2_asymptotic_v17-v1/MINIAODSIM",
"/ZZ_TuneCP5_13TeV-pythia8/RunIISummer20UL16MiniAODv2-106X_mcRun2_asymptotic_v17-v1/MINIAODSIM",
"/TTGG_TuneCP5_13TeV-amcatnlo-pythia8/RunIISummer20UL16MiniAODv2-106X_mcRun2_asymptotic_v17-v2/MINIAODSIM",
"/TTGJets_TuneCP5_13TeV-amcatnloFXFX-madspin-pythia8/RunIISummer20UL16MiniAODv2-106X_mcRun2_asymptotic_v17-v1/MINIAODSIM",
"/TGJets_TuneCP5_13TeV-amcatnlo-madspin-pythia8/RunIISummer20UL16MiniAODv2-106X_mcRun2_asymptotic_v17_ext1-v2/MINIAODSIM",
"/TTJets_TuneCP5_13TeV-amcatnloFXFX-pythia8/RunIISummer20UL16MiniAODv2-106X_mcRun2_asymptotic_v17-v1/MINIAODSIM"

2017

 "/ZGTo2NuG_TuneCP5_13TeV-amcatnloFXFX-pythia8/RunIISummer20UL17MiniAODv2-106X_mc2017_realistic_v9-v2/MINIAODSIM"

2018

"/ZGTo2NuG_TuneCP5_13TeV-amcatnloFXFX-pythia8/RunIISummer20UL18MiniAODv2-106X_upgrade2018_realistic_v16_L1v1-v2/MINIAODSIM",
"/ZGToLLG_01J_5f_TuneCP5_13TeV-amcatnloFXFX-pythia8/RunIISummer20UL18MiniAODv2-106X_upgrade2018_realistic_v16_L1v1-v1/MINIAODSIM",
"/WGToLNuG_01J_5f_TuneCP5_13TeV-amcatnloFXFX-pythia8/RunIISummer20UL18MiniAODv2-106X_upgrade2018_realistic_v16_L1v1-v1/MINIAODSIM",
"/WW_TuneCP5_13TeV-pythia8/RunIISummer20UL18MiniAODv2-106X_upgrade2018_realistic_v16_L1v1-v1/MINIAODSIM",
"/WZ_TuneCP5_13TeV-pythia8/RunIISummer20UL18MiniAODv2-106X_upgrade2018_realistic_v16_L1v1-v1/MINIAODSIM",
"/ZZ_TuneCP5_13TeV-pythia8/RunIISummer20UL18MiniAODv2-106X_upgrade2018_realistic_v16_L1v1-v2/MINIAODSIM",
"/TTGG_TuneCP5_13TeV-amcatnlo-pythia8/RunIISummer20UL18MiniAODv2-106X_upgrade2018_realistic_v16_L1v1-v2/MINIAODSIM",
"/TTGJets_TuneCP5_13TeV-amcatnloFXFX-madspin-pythia8/RunIISummer20UL18MiniAODv2-106X_upgrade2018_realistic_v16_L1v1_ext1-v1/MINIAODSIM",
"/TGJets_TuneCP5_13TeV-amcatnlo-madspin-pythia8/RunIISummer20UL18MiniAODv2-106X_upgrade2018_realistic_v16_L1v1-v2/MINIAODSIM",
"/TTJets_TuneCP5_13TeV-amcatnloFXFX-pythia8/RunIISummer20UL18MiniAODv2-106X_upgrade2018_realistic_v16_L1v1-v2/MINIAODSIM"
emanueledimarco commented 1 year ago

@youyingli ok I'll do the fixes you say, and produce the list of MicroAODs that you say. In addition, I am going to make the following DoubleEMEnriched samples, which are needed by Badder @bmarzocc :

/QCD_Pt-30to40_DoubleEMEnriched_MGG-80toInf_TuneCP5_13TeV-pythia8/RunIISummer20UL16MiniAODAPVv2-106X_mcRun2_asymptotic_preVFP_v11-v1/MINIAODSIM
/QCD_Pt-30to40_DoubleEMEnriched_MGG-80toInf_TuneCP5_13TeV-pythia8/RunIISummer20UL16MiniAODv2-106X_mcRun2_asymptotic_v17-v1/MINIAODSIM
/QCD_Pt-30to40_DoubleEMEnriched_MGG-80toInf_TuneCP5_13TeV-pythia8/RunIISummer20UL17MiniAODv2-106X_mc2017_realistic_v9-v1/MINIAODSIM
/QCD_Pt-30to40_DoubleEMEnriched_MGG-80toInf_TuneCP5_13TeV-pythia8/RunIISummer20UL18MiniAODv2-106X_upgrade2018_realistic_v16_L1v1-v1/MINIAODSIM
/QCD_Pt-30toInf_DoubleEMEnriched_MGG-40to80_TuneCP5_13TeV-pythia8/RunIISummer20UL16MiniAODAPVv2-106X_mcRun2_asymptotic_preVFP_v11-v2/MINIAODSIM
/QCD_Pt-30toInf_DoubleEMEnriched_MGG-40to80_TuneCP5_13TeV-pythia8/RunIISummer20UL16MiniAODv2-106X_mcRun2_asymptotic_v17-v2/MINIAODSIM
/QCD_Pt-30toInf_DoubleEMEnriched_MGG-40to80_TuneCP5_13TeV-pythia8/RunIISummer20UL17MiniAODv2-106X_mc2017_realistic_v9-v2/MINIAODSIM
/QCD_Pt-30toInf_DoubleEMEnriched_MGG-40to80_TuneCP5_13TeV-pythia8/RunIISummer20UL18MiniAODv2-106X_upgrade2018_realistic_v16_L1v1-v2/MINIAODSIM
/QCD_Pt-40ToInf_DoubleEMEnriched_MGG-80ToInf_TuneCP5_13TeV-pythia8/RunIISummer20UL16MiniAODAPVv2-106X_mcRun2_asymptotic_preVFP_v11-v2/MINIAODSIM
/QCD_Pt-40ToInf_DoubleEMEnriched_MGG-80ToInf_TuneCP5_13TeV-pythia8/RunIISummer20UL16MiniAODv2-106X_mcRun2_asymptotic_v17-v2/MINIAODSIM
/QCD_Pt-40ToInf_DoubleEMEnriched_MGG-80ToInf_TuneCP5_13TeV-pythia8/RunIISummer20UL17MiniAODv2-106X_mc2017_realistic_v9-v2/MINIAODSIM
/QCD_Pt-40ToInf_DoubleEMEnriched_MGG-80ToInf_TuneCP5_13TeV-pythia8/RunIISummer20UL18MiniAODv2-106X_upgrade2018_realistic_v16_L1v1-v2/MINIAODSIM
emanueledimarco commented 1 year ago

@youyingli and others,

I have updated almost all the catalogs with all new samples requested after running the check step. Still running the check step for some large samples of the RunIISummer20UL18 catalog. As soon as that is done I will commit it and, if everything seems fine, the PR can be merged, I guess.

At least people can start using the 2016-2017 catalogs already now. @rohithsaradhy

emanueledimarco commented 1 year ago

@youyingli et al. I have finally updated the catalogs for RunIISummer20UL18 campaign also. This should be everything I had in my backlog (sorry if it took so long, but I had to do it interactively, since in batch via condor it was not working properly, at least for me).

@bmarzocc the catalogs also include the DoubleEMEnriched QCD samples that you asked to produce.

Let me know if you see something not correct, otherwise this I think can be merged.

youyingli commented 1 year ago

@emanueledimarco, thanks for large contribution to the MicroAOD productions. However, I found out that the following

/EGamma/phys_higgs-RunIISummer20UL18-legacyRunII-v0-Run2018A-UL2018_MiniAODv2-v1-a7c88d6f8b657df10cc9b059890284c0/USER
/EGamma/phys_higgs-RunIISummer20UL18-legacyRunII-v0-Run2018B-UL2018_MiniAODv2-v1-a7c88d6f8b657df10cc9b059890284c0/USER
/EGamma/phys_higgs-RunIISummer20UL18-legacyRunII-v0-Run2018C-UL2018_MiniAODv2-v1-a7c88d6f8b657df10cc9b059890284c0/USER
/EGamma/phys_higgs-RunIISummer20UL18-legacyRunII-v0-Run2018D-UL2018_MiniAODv2-v2-a7c88d6f8b657df10cc9b059890284c0/USER

are accidentally added into RunIISummer20UL18. Could you remove them? Then, the PR can be merged.

On the other hand, RunIISummer20UL18 is missing the microAOD of

/TTJets_TuneCP5_13TeV-amcatnloFXFX-pythia8/RunIISummer20UL18MiniAODv2-106X_upgrade2018_realistic_v16_L1v1-v2/MINIAODSIM Is that only the issue from HTConder or it's really missing during MicroAOD production? If the latter case, I can merge this PR when you remove 0 event EGamma above, and missing TTJets can be added at the next PR.

emanueledimarco commented 1 year ago

@youyingli I removed the EGamma datasets from the RunIISummer20UL18 catalog. For the TTJets sample, I don't find it in DAS, so maybe was not produced. I can produce it now, but maybe better close this PR first. Thanks.

youyingli commented 1 year ago

@emanueledimarco , thanks but do you mean "close" is merged ？

rohithsaradhy commented 1 year ago

@youyingli and others,

I have updated almost all the catalogs with all new samples requested after running the check step. Still running the check step for some large samples of the RunIISummer20UL18 catalog. As soon as that is done I will commit it and, if everything seems fine, the PR can be merged, I guess.

At least people can start using the 2016-2017 catalogs already now. @rohithsaradhy

Hi @youyingli @emanueledimarco @rchatter @Xingfu23,, The main missing samples are the following :

For UL16postVFP

WHiggs0L1ToGG: /WHiggs0L1ToGG_M125_TuneCP5_13TeV-JHUGenV7011-pythia8/RunIISummer20UL16MiniAODv2-106X_mcRun2_asymptotic_v17-v2/MINIAODSIM
WHiggs0MToGG:  /WHiggs0MToGG_M125_TuneCP5_13TeV-JHUGenV7011-pythia8/RunIISummer20UL16MiniAODv2-106X_mcRun2_asymptotic_v17-v2/MINIAODSIM
WHiggs0Mf05ph0ToGG: /WHiggs0Mf05ph0ToGG_M125_TuneCP5_13TeV-JHUGenV7011-pythia8/RunIISummer20UL16MiniAODv2-106X_mcRun2_asymptotic_v17-v2/MINIAODSIM

Perhaps @Xingfu23 can confirm this.

Thanks, Rohith

Xingfu23 commented 1 year ago

Hi @youyingli @emanueledimarco @rchatter and @rohithsaradhy I also checked the missing samples again these three samples listed by @rohithsaradhy are the missing ac samples for UL16postVFP

Thanks, Xingfu

emanueledimarco commented 1 year ago

@emanueledimarco , thanks but do you mean "close" is merged ？

yes, I meant "merged", sorry

emanueledimarco commented 1 year ago

The main missing samples are the following :

I can produce the microAODs for these, and add these in a future PR to finish this one that is hanging for so long time

emanueledimarco commented 1 year ago

Hi @youyingli @emanueledimarco @rchatter and @rohithsaradhy I also checked the missing samples again these three samples listed by @rohithsaradhy are the missing ac samples for UL16postVFP

Thanks, Xingfu

@rchatter @rohithsaradhy since I had everything set up already I started the jobs over crab for those three samples

rohithsaradhy commented 1 year ago

Hi @youyingli @emanueledimarco @rchatter and @rohithsaradhy I also checked the missing samples again these three samples listed by @rohithsaradhy are the missing ac samples for UL16postVFP Thanks, Xingfu

@rchatter @rohithsaradhy since I had everything set up already I started the jobs over crab for those three samples

Many thanks!

Xingfu23 commented 1 year ago

Hi @emanueledimarco and @youyingli,

I found that mc samples for the diphoton background for 2016 and 2018 are missing. This background is one of the major backgrounds in vhmet analysis. I will list the samples following, please help me with this:

For UL16PreVFP

/DiPhotonJetsBox_MGG-80toInf_13TeV-sherpa/RunIISummer20UL16MiniAODAPVv2-106X_mcRun2_asymptotic_preVFP_v11-v2/MINIAODSIM

For UL16PostVFP

/DiPhotonJetsBox_MGG-80toInf_13TeV-sherpa/RunIISummer20UL16MiniAODv2-106X_mcRun2_asymptotic_v17-v2/MINIAODSIM

For UL18

/DiPhotonJetsBox_MGG-80toInf_13TeV-sherpa/RunIISummer20UL18MiniAODv2-106X_upgrade2018_realistic_v16_L1v1-v2/MINIAODSIM

Many Thanks Xingfu

emanueledimarco commented 1 year ago

Hi @Xingfu23 I have added these to the production that I was running for the other few samples. I'll do a unique PR with the others when I have updated the catalogs/