search

key4hep / k4FWCore

Core Components for the Gaudi-based Key4hep Framework
Apache License 2.0
10 stars 26 forks source link

Further issues with k4run #141

Closed Zehvogel closed 1 year ago

Zehvogel commented 1 year ago

See discussion in: https://github.com/key4hep/k4FWCore/pull/134

Two further issues popped up:

giovannimarchiori commented 1 year ago

Hi @Zehvogel this is how I setup the environment

export FCCBASEDIR=$PWD
source /cvmfs/sw-nightlies.hsf.org/key4hep/setup.sh                                                                                                                                                            

export K4RECCALORIMETER=$PWD/k4RecCalorimeter/install/share/k4RecCalorimeter
export K4SIMGEANT4=$PWD/k4SimGeant4/install/share/k4SimGeant4
export FCCDETECTORS=$PWD/FCCDetectors/

# with K4WCORE                                                                                                                                                                                                  
export K4FWCORE=$PWD/k4FWCore/install/share/k4FWCore
export PATH=$PWD/k4RecCalorimeter/install/bin/:$PWD/k4FWCore/install/bin/:$PWD/k4SimGeant4/install/bin:$PATH
export CMAKE_PREFIX_PATH=$PWD/k4RecCalorimeter/install:$PWD/k4FWCore/install/:$PWD/k4SimGeant4/install/:$PWD/FCCDetectors/install/:$CMAKE_PREFIX_PATH
export LD_LIBRARY_PATH=$PWD/k4RecCalorimeter/install/lib:$PWD/k4RecCalorimeter/install/lib64:$PWD/k4FWCore/install/lib:$PWD/k4SimGeant4/install/lib:$PWD/k4SimGeant4/install/lib64:$PWD/FCCDetectors/install/lib:$PWD/FCCDetectors/install/lib64:$LD_LIBRARY_PATH
export PYTHONPATH=$PWD/k4RecCalorimeter/install/python:$PWD/k4FWCore/install/python:$PWD/k4SimGeant4/install/python:$PYTHONPATH

then make a fresh build and run with 

cd LAr_scripts/FCCSW_ecal/
fccrun runTopoAndSlidingWindowAndCaloSim.py

Output:

/home/gmarchio/work/fcc-main/FCCDetectors/
Traceback (most recent call last):
  File "/home/gmarchio/work/fcc-main/k4FWCore/install/bin/k4run", line 131, in <module>
    add_arguments(parser, ApplicationMgr())
  File "/home/gmarchio/work/fcc-main/k4FWCore/install/bin/k4run", line 65, in add_arguments
    parser.add_argument( f"--{propName}", f"--{propNameReversed}", type=proptype, help=props[prop][1],
  File "/cvmfs/sw-nightlies.hsf.org/key4hep/releases/2023-06-24/x86_64-almalinux9-gcc11.3.1-opt/python/3.10.10-4lh2y5/lib/python3.10/argparse.py", line 1441, in add_argument
    return self._add_action(action)
  File "/cvmfs/sw-nightlies.hsf.org/key4hep/releases/2023-06-24/x86_64-almalinux9-gcc11.3.1-opt/python/3.10.10-4lh2y5/lib/python3.10/argparse.py", line 1807, in _add_action
    self._optionals._add_action(action)
  File "/cvmfs/sw-nightlies.hsf.org/key4hep/releases/2023-06-24/x86_64-almalinux9-gcc11.3.1-opt/python/3.10.10-4lh2y5/lib/python3.10/argparse.py", line 1643, in _add_action
    action = super(_ArgumentGroup, self)._add_action(action)
  File "/cvmfs/sw-nightlies.hsf.org/key4hep/releases/2023-06-24/x86_64-almalinux9-gcc11.3.1-opt/python/3.10.10-4lh2y5/lib/python3.10/argparse.py", line 1455, in _add_action
    self._check_conflict(action)
  File "/cvmfs/sw-nightlies.hsf.org/key4hep/releases/2023-06-24/x86_64-almalinux9-gcc11.3.1-opt/python/3.10.10-4lh2y5/lib/python3.10/argparse.py", line 1592, in _check_conflict
    conflict_handler(action, confl_optionals)
  File "/cvmfs/sw-nightlies.hsf.org/key4hep/releases/2023-06-24/x86_64-almalinux9-gcc11.3.1-opt/python/3.10.10-4lh2y5/lib/python3.10/argparse.py", line 1601, in _handle_conflict_error
    raise ArgumentError(action, message % conflict_string)
argparse.ArgumentError: argument --SimG4Svc.SimG4FullSimActions.OutputLevel/--OutputLevel.SimG4Svc.SimG4FullSimActions: conflicting option strings: --SimG4Svc.SimG4FullSimActions.OutputLevel, --OutputLevel.SimG4Svc.SimG4FullSimActions
Zehvogel commented 1 year ago

Looks like SimG4FullSimActions('SimG4Svc.SimG4FullSimActions') exists more than once in ApplicationMgr().allConfigurables.values()

(FCCDataSvc('EventDataSvc'), GenAlg('GenAlg'), MomentumRangeParticleGun('ToolSvc.ParticleGun_Electron'), MomentumRangeParticleGun('GenAlg.ParticleGun_Electron'), HepMCToEDMConverter('HepMCToEDMConverter'), GeoSvc('GeoSvc'), SimG4FullSimActions('SimG4Svc.SimG4FullSimActions'), SimG4FullSimActions('SimG4Svc.SimG4FullSimActions'), SimG4Svc('SimG4Svc'), SimG4ConstantMagneticFieldTool('ToolSvc.SimG4ConstantMagneticFieldTool'), SimG4SaveCalHits('ToolSvc.saveECalBarrelHits'), SimG4SaveCalHits('ToolSvc.saveECalEndcapHits'), SimG4SaveCalHits('ToolSvc.saveHCalBarrelHits'), SimG4PrimariesFromEdmTool('ToolSvc.EdmConverter'), SimG4PrimariesFromEdmTool('SimG4Alg.EdmConverter'), SimG4Alg('SimG4Alg'), CalibrateInLayersTool('ToolSvc.CalibrateECalBarrel'), CalibrateCaloHitsTool('ToolSvc.CalibrateHCal'), CalibrateCaloHitsTool('ToolSvc.CalibrateECalEndcap'), CalibrateCaloHitsTool('ToolSvc.CalibrateHCalEndcap'), CalibrateInLayersTool('CreateECalBarrelCellsStep1.CalibrateECalBarrel'), CreateCaloCells('CreateECalBarrelCellsStep1'), RedoSegmentation('ReSegmentationEcal'), CreateCaloCells('CreateECalBarrelCells'), CellPositionsECalBarrelTool('ToolSvc.CellPositionsECalBarrel'), CreateCaloCellPositionsFCCee('ECalBarrelPositionedCells'), CalibrateCaloHitsTool('CreateHCaloCells.CalibrateHCal'), CreateCaloCells('CreateHCaloCells'), CalibrateCaloHitsTool('CreateEcalEndcapCaloCells.CalibrateECalEndcap'), CreateCaloCells('CreateEcalEndcapCaloCells'), CreateEmptyCaloCellsCollection('CreateEmptyCaloCells'), CaloTowerTool('ToolSvc.towers'), CreateCaloClustersSlidingWindow('CreateClusters'), CreateCaloCellPositionsFCCee('ECalBarrelPositionedCaloClusterCells'), CorrectCaloClusters('correctCaloClusters'), CaloTopoClusterInputTool('ToolSvc.CreateTopoInput'), TopoCaloNeighbours('ToolSvc.ReadNeighboursMap'), TopoCaloNoisyCells('ToolSvc.ReadNoisyCellsMap'), CaloTopoClusterInputTool('CreateTopoClusters.CreateTopoInput'), TopoCaloNeighbours('CreateTopoClusters.ReadNeighboursMap'), TopoCaloNoisyCells('CreateTopoClusters.ReadNoisyCellsMap'), CellPositionsECalBarrelTool('CreateTopoClusters.CellPositionsECalBarrel'), CaloTopoClusterFCCee('CreateTopoClusters'), CreateCaloCellPositionsFCCee('ECalBarrelPositionedCaloTopoClusterCells'), CorrectCaloClusters('correctCaloTopoClusters'), PodioOutput('out'), ChronoAuditor('ChronoAuditor'), AuditorSvc('AuditorSvc'), EventCounter('event_counter'), ApplicationMgr('ApplicationMgr'))

I updated https://github.com/key4hep/k4FWCore/pull/140 which should restore the old behaviour for now.

However, I think we should investigate if the duplication itself is not a bug. Any thoughts on that @BrieucF, @kjvbrt?

BrieucF commented 1 year ago

Hi, not sure whether it is a bug or not but the duplication seems to be due to the fact that all tools declared in the steering file (even if not used by any algorithm) are picked up by the ApplicationMgr. When passed to e.g. SimG4Svc in this case, it is somehow picked up a second time. An additional piece of information is that this seems to happen only because SimG4Svc is passed as an ExtSvc of ApplicationMgr (tools passed to algorithm belonging to TopAlg appear only once in app_mgr.allConfigurables.values()).

Zehvogel commented 1 year ago

@giovannimarchiori can you confirm if https://github.com/key4hep/k4FWCore/pull/140 solves your issues now?

giovannimarchiori commented 1 year ago

Hi @Zehvogel yes, #140 solves the issues Thanks, Giovanni