search

e4nu / emMCRadCorr

This is a package that can be used to add radiative corrections to any MC event generator. It is developed oriented to neutrino event generators running on electron-scattering mode. The package modifies the output of neutrino event generators in NuHEPMC format. It can also be used to modify GENIE gst files.
GNU Affero General Public License v3.0
0 stars 1 forks source link

Change FNAL script #1

Closed jtenavidal closed 4 months ago

jtenavidal commented 6 months ago
jtenavidal commented 5 months ago

Change -f option. Must copy setup files to pnfs, otherwise it cannot find it

/pnfs/genie/scratch/users/jtenavid/GENIE_e4nu_Generations/2024Generation/Test_newHepMC3_14Q2

<serial>                                                                                                                                                          
jobsub_submit  -n --memory=4 --disk=4 --expected-lifetime=4h  --OS=SL7 --lines '+FERMIHTC_AutoRelease=True' -f /pnfs/genie/scratch/users/jtenavid/GENIE_e4nu_Gene\
rations/2024Generation/Test_newHepMC3_14Q2/setup_FNAL.sh -f /pnfs/genie/scratch/users/jtenavid/GENIE_e4nu_Generations/2024Generation/Test_newHepMC3_14Q2/setup_GE\
NIE.sh --lines '+FERMIHTC_GraceMemory=4096' --lines '+FERMIHTC_GraceLifetime=6000' --mail_on_error --singularity-image /cvmfs/singularity.opensciencegrid.org/fer\
milab/fnal-wn-sl7:latest  file:///pnfs/genie/scratch/users/jtenavid/GENIE_e4nu_Generations/2024Generation/Test_newHepMC3_14Q2//master-routine_validation_01-eScat\
tering/e_on_1000010010_0.sh                                                                                                                                       
</serial>
<serial>

and something else is off?

jtenavidal commented 5 months ago

Also need to add option to submit flux in the path

jtenavidal commented 5 months ago 
python submit_rad_GENIE_gpvm.py --directory /pnfs/genie/scratch/users/jtenavid/GENIE_e4nu_Generations/2024Generation/Test_NewHepmC --ebeam-energy 4.325 --model vanderhaghen --target 1000010010 --genie-git-location https://github.com/e4nu/Generator-NuHepMC.git --genie-git-branch hepmc3-Q214 --tune G18_10a_00_000 --xsec /pnfs/genie/persistent/users/jtenavid/e4nu_files/GENIE_Files/2024Splines/G18_10a_Dipole_Q2_14_eFreeNucleon.xml --nevents 100 --gst-output --no-ghep-output
jtenavidal commented 5 months ago 
<serial>                                                                                                                                                                                     
jobsub_submit  -n --memory=2GB --disk=2GB --expected-lifetime=4h  --OS=SL7 --lines '+FERMIHTC_AutoRelease=True' -f /pnfs/genie/scratch/users/jtenavid/GENIE_e4nu_Generations/2024Generation/\
Caleb_Rarita//setup_FNAL.sh -f /pnfs/genie/scratch/users/jtenavid/GENIE_e4nu_Generations/2024Generation/Caleb_Rarita//setup_GENIE.sh --lines '+FERMIHTC_GraceMemory=4096' --lines '+FERMIHTC\
_GraceLifetime=6000' --mail_on_error --singularity-image /cvmfs/singularity.opensciencegrid.org/fermilab/fnal-wn-sl7:latest  file:///pnfs/genie/scratch/users/jtenavid/GENIE_e4nu_Generation\
s/2024Generation/Test_newHepMC3_14Q2//master-routine_validation_01-eScattering/e_on_1000010010_0.sh                                                                                          
</serial>

Errors:

Use job id 12136322.0@jobsub04.fnal.gov to retrieve output

jtenavidal commented 5 months ago

54572173.0@jobsub03.fnal.gov

jtenavidal commented 5 months ago 
Tue Apr 30 07:54:36 UTC 2024 54572174.0@jobsub03.fnal.gov BEGIN EXECUTION rad_corr_e_on_1000010010_0.sh
Warning: Unused class rule: std::vector<HepMC::GenVertexData>
Warning: Unused class rule: std::vector<HepMC::GenParticleData>
In file included from /cvmfs/larsoft.opensciencegrid.org/products/root/v6_26_06b/Linux64bit+3.10-2.17-e26-p3913-prof/include/TString.h:29:0,
                 from /cvmfs/larsoft.opensciencegrid.org/products/root/v6_26_06b/Linux64bit+3.10-2.17-e26-p3913-prof/include/TNamed.h:26,
                 from /cvmfs/larsoft.opensciencegrid.org/products/root/v6_26_06b/Linux64bit+3.10-2.17-e26-p3913-prof/include/TDirectory.h:24,
                 from /cvmfs/larsoft.opensciencegrid.org/products/root/v6_26_06b/Linux64bit+3.10-2.17-e26-p3913-prof/include/TDirectoryFile.h:25,
                 from /cvmfs/larsoft.opensciencegrid.org/products/root/v6_26_06b/Linux64bit+3.10-2.17-e26-p3913-prof/include/TFile.h:28,
                 from /srv/no_xfer/0/TRANSFERRED_INPUT_FILES/emMCRadCorr/build/_deps/hepmc3-src/rootIO/src/WriterRoot.cc:16:
/cvmfs/larsoft.opensciencegrid.org/products/root/v6_26_06b/Linux64bit+3.10-2.17-e26-p3913-prof/include/ROOT/RStringView.hxx:25:23: fatal error: string_view: No such file or directory
 #include <string_view>
                       ^
compilation terminated.
make[2]: *** [_deps/hepmc3-build/rootIO/CMakeFiles/HepMC3rootIO.dir/src/WriterRoot.cc.o] Error 1
make[1]: *** [_deps/hepmc3-build/rootIO/CMakeFiles/HepMC3rootIO.dir/all] Error 2
make: *** [all] Error 2
Copying 253144585 bytes https://fndcadoor.fnal.gov:2880/genie/scratch/users/jtenavid/GENIE_e4nu_Generations/2024Generation/Test_NewHepmC/master-routine_validation_01-eScattering/e_on_1000010010_0.hepmc3 => file:///srv/no_xfer/0/TRANSFERRED_INPUT_FILES/e_on_1000010010_0.hepmc3
./rad_corr_e_on_1000010010_0.sh: line 9: ./process_radcorr: No such file or directory
gfal-copy error: 2 (No such file or directory) - Could not stat the source: errno reported by local system call No such file or directory
Perhaps you forgot a -D to indicate destination is a directory? 
gfal-copy error: 2 (No such file or directory) - Could not stat the source: errno reported by local system call No such file or directory
Perhaps you forgot a -D to indicate destination is a directory? 
cvmfs info:
attr_get: No such file or directory
Could not get "logbuffer" for /cvmfs/genie.opensciencegrid.org/
Tue Apr 30 07:58:07 UTC 2024 rad_corr_e_on_1000010010_0.sh COMPLETED with exit status 2
jtenavidal commented 5 months ago

singularity or apptainer

apptainer shell /cvmfs/singularity.opensciencegrid.org/fermilab/fnal-wn-sl7:latest
jtenavidal commented 5 months ago

https://computing.fnal.gov/wilsoncluster/containers/

https://mu2ewiki.fnal.gov/wiki/Grids

jtenavidal commented 5 months ago

-> Give direction to build code ? tar -zcvf emMCRadCorr_build.tgz build/

bring in the tarball you created before with caches and large files excluded:

xrdcp -s root://cmseos.fnal.gov//store/user/username/emMCRadCorr_build.tgz .

tar -xf CMSSW_10_6_4.tgz rm CMSSW_10_6_4.tgz cd CMSSW_10_6_4/src/

https://uscms.org/uscms_at_work/computing/setup/batch_systems.shtml

jtenavidal commented 5 months ago 
2024-05-03 09:31:17 CDT - Vito Di Benedetto (Additional comments (customer communication))
Hi Julia,
yes, jobsub_lite allow users to pass a tarball to subjobs, this can be done by using the option:
--tar-file-name dropbox:///x/y/z.tar
to the main submission command of the DAG, subjobs will have access to it.
More details about jobsub_lite are available here: https://fifewiki.fnal.gov/wiki/Jobsub_Lite
and about details for File Transfers in jobsub lite, the documentation has a link to: https://github.com/fermitools/jobsub_lite/wiki/File-Transfers-in-jobsub-lite

Another node about your submission command, I see you have the option:
--OS=SL7
this is used to select the bare metal OS, but because you are requesting your jobs to run in a SL7 container, specifically cvmfs/singularity.opensciencegrid.org/fermilab/fnal-wn-sl7:latest
the OS is choose through the container, so you are not limited to require worker nodes with SL7 installed, but you job can run on any worker node no mater their OS.

I hope this helps,
Vito
jtenavidal commented 5 months ago

Add 

source /cvmfs/fermilab.opensciencegrid.org/products/genie/bootstrap_genie_ups.sh
source /cvmfs/fermilab.opensciencegrid.org/products/common/etc/setups
setup ifdhc v2_6_6

in radcorr script

jtenavidal commented 4 months ago 
python submit_rad_GENIE_gpvm.py --directory /pnfs/genie/scratch/users/jtenavid/GENIE_e4nu_Generations/2024Generation/Paper_New/ --ebeam-energy 4.325 --model vanderhaghen --target 1000010010 --genie-git-location https://github.com/e4nu/Generator-NuHepMC.git --genie-git-branch hepmc3-Q214 --tune G18_10a_00_000 --xsec /pnfs/genie/persistent/users/jtenavid/e4nu_files/GENIE_Files/2024Splines/G18_10a_Dipole_Q2_14_eFreeNucleon.xml --nevents 100 --gst-output --no-ghep-output --git-tarball-location /genie/app/users/jtenavid/Software/e4v/E4NuAnalysis/Source/emMCRadCorr --config-dir /pnfs/genie/persistent/users/jtenavid/e4nu_files/e4nu-config-files/e4nu-GENIE-config/ --event-gen-list EMQE