search

cernopendata / opendata.cern.ch

Source code for the CERN Open Data portal
http://opendata.cern.ch/
GNU General Public License v2.0
666 stars 148 forks source link

CMS: import the generator fragments #2698

Open katilp opened 5 years ago

katilp commented 5 years ago

For generator fragments, we link directly to mcm. Example: http://opendata-dev.web.cern.ch/record/808# where Generator parameters link brings directly to https://cms-pdmv.cern.ch/mcm/public/restapi/requests/get_fragment/FSQ-LowPU2010-00013

This is fine for the moment but for long-term preservation, we should import the contents.

katilp commented 8 months ago

The link has also changed, but for the moment, redirecting to the new link works. For example, the one above opens now in https://cms-pdmv-prod.web.cern.ch/mcm/public/restapi/requests/get_fragment/FSQ-LowPU2010-00013

Right now (before adding the 2016 MC), there are 2085 of these (in the "url" json field of the provenance section)

$ grep cms-pdmv.cern.ch/mcm/public/restapi cernopendata/modules/fixtures/data/records/cms*json | grep '"url":' |
 wc -l
2085
katilp commented 1 week ago

@tiborsimko @jmhogan @tpmccauley We should have a subdirectory for these fragments and copy them over. We have /eos/opendata/cms/configuration-files/ but that's not the best place. Some of these fragments can be shared between MC of different years so I would have them in a new separate directory, e.g. /eos/opendata/cms/fragments/ or /eos/opendata/cms/generator-fragments/ Your thoughts?

jmhogan commented 1 week ago

@katilp Agreed, that would be a good folder name, I like generator-fragments best

d0leh commented 5 days ago

I have created generator-fragments directory in /eos/opendata/cms/upload/mohammad/, where it has all the fragments.

It currently has 2058 Python scripts, as 27 (2085 - 2058) fragments have occurred more than once and were overwritten with the same code.