out of the box dirac troubles

[rob-c]

I was hoping to use ganga to test file storage here at ECDF using the standard gridpp tools. (plan was to test the DiracFile.put to Edinburgh to test if the xrootd storage protocol is working/supported)

Unfortunately I'm seeing the following problems:

From: /cvmfs/ganga.cern.ch/runGanga-dirac.sh:

[11:15:47]
Ganga In [4]: j=Job();j.backend=Dirac();j.submit()
INFO     submitting job 0
INFO     job 0 status changed to "submitting"
INFO     Preparing Executable application.
INFO     Created shared directory: conf-27680420-c610-442b-9d49-627550c0d2c9
INFO     Preparing subjobs
INFO     Setting Command to be: '['echo', 'Hello World']'
INFO     submitting job 0 to Dirac backend
WARNING  Required credential [DiracProxy(group='gridpp_user', encodeDefaultProxyFileName=True)] not found in store
ERROR    'DiracEnvSource' config variable empty
/bin/sh: dirac-proxy-init: command not found
/bin/sh: dirac-proxy-init: command not found
/bin/sh: dirac-proxy-init: command not found
/bin/sh: dirac-proxy-init: command not found
/bin/sh: dirac-proxy-init: command not found
ERROR    CredentialsError: Proxy object cannot satisfy its own requirements
ERROR    JobManagerError: error during submit
ERROR    JobManagerError: error during submit ... reverting job 0 to the new status
INFO     job 0 status changed to "new"
ERROR    JobError: Error: JobManagerError: error during submit

Also this fails to launch out of the box when .gangarc doesn't exist due to:

-bash-4.1$ /cvmfs/ganga.cern.ch/runGanga-dirac.sh

*** Welcome to Ganga ***
Version: 6.6.4
Documentation and support: http://cern.ch/ganga
Type help() or help('index') for online help.

This is free software (GPL), and you are welcome to redistribute it
under certain conditions; type license() for details.

INFO     reading config file /cvmfs/ganga.cern.ch/Dirac.ini
ERROR    problems with loading Plugin /cvmfs/ganga.cern.ch/Ganga/install/6.6.4/python/GangaDirac
ERROR    Reason: GangaValueError: DIRAC Proxy `group` is not set. Set this in ~/.gangarc in `[defaults_DiracProxy]/group`
PluginError: Failed to load plugin: /cvmfs/ganga.cern.ch/Ganga/install/6.6.4/python/GangaDirac. Ganga will now shutdown to prevent job corruption.
(consider --debug option for more information)
INFO     Stopping Job processing before shutting down Repositories
INFO     Shutting Down Ganga Repositories
INFO     Registry Shutdown

I'll check with lsst to see what they've done in the past but this is not ideal as they now probably have some undocumented solution and in principle on gridpp we don't have an out of the box working hello world style job.

I think I expect the following... /cvmfs/ganga.cern.ch/runGanga.sh

[11:23:08]
Ganga In [1]: j=Job();j.backend=Dirac();j.submit()
ERROR    name 'Dirac' is not defined

[alexanderrichards]

I couldn't agree more! In fact, I believe I have mentioned this a few times in the past. The out of the box behavior is not very Dirac friendly. As I recall you have to have a config with the RuntimePath set to dirac and the Dirac section already present and pointing to the Dirac bashrc and then you also need to set the defaults_DiracProxy->group attribute.

[rob-c]

@alexanderrichards I'll check the current docs for how to get a VO certificate but if the current route is for everyone to go via getting a gridpp_user account then we should strongly consider having all of the defaults in place to be able to submit a hello world job with these credentials. I'll discuss this with a few people off list here. I'll get in touch with the LSST users and see what their current configuration is and we should consider adding approved configs to cvmfs as reference (after some thought).

For now I'll drop back to using DIRAC natively but it would be nice if this just worked so I'll try and open some dialogue with LSST people on improving the gridpp docs.

[rob-c]

(After a few minutes of poking at this I'm seeing some strange behaviour compared to how I was expecting some things to work so I'd encourage someone checking the latest dev branch for gridpp dirac support in 6.7.0 before it's released).

[rob-c]

I'm also seeing:

Ganga In [2]: j=Job();j.backend=Dirac();j.submit()
INFO     submitting job 8
INFO     job 8 status changed to "submitting"
INFO     Preparing Executable application.
INFO     Created shared directory: conf-f3f53cb0-6866-4d89-b62b-13f7fd55f681
ERROR    ConfigError: attempt to create a configuration section [defaults_GaudiXMLSummary] after bootstrap
ERROR    ConfigError: attempt to create a configuration section [defaults_GaudiXMLSummary] after bootstrap  ... reverting job 8 to the new status

(Submitting to the local backend works as expected and job completes so will pass tests).

This is from a clean gridpp DIRAC instance with no LHCb ganga environment variables set so I suspect there could be a broken dependency somewhere within the share directory code which is importing Gaudi/LHCb incorrectly on vanilla DIRAC. (This could be my environment being polluted but I'm fairly sure it's not in this case)

[alexanderrichards]

That's a new one on me. But then I haven't tried it in a few weeks and there has been a lot go in since then so I'm afraid I can't say if it's just your environment or not. If I get a chance I'll try again today. Either way I think that it proves the point that things aren't behaving as seamlessly as they should.

[rob-c]

@alexanderrichards No major rush on the last point it could be some left over thing somewhere in a config file I've just forgotten about, if submitting a gridpp Dirac job works on dev this is just an out of the box config issue