search

dmwm / CRABClient

runrange
14 stars 36 forks source link

Client eats important tracebacks #4481

Open PerilousApricot opened 9 years ago

PerilousApricot commented 9 years ago 
DEBUG 2015-05-18 23:38:19,053:   Executing command: 'submit'                    
DEBUG 2015-05-18 23:38:19,053:   Found CRAB cache file /home/meloam/.crab3      
DEBUG 2015-05-18 23:38:19,063:   Checking credentials                           
DEBUG 2015-05-18 23:38:19,064:   Found proxy file /tmp/x509up_u112870           
DEBUG 2015-05-18 23:38:19,064:   Getting proxy life time left                   
DEBUG 2015-05-18 23:38:19,166:   Executing :                                    
 command : voms-proxy-info -file /tmp/x509up_u112870 -timeleft                  
 output : 85755                                                                 

 error:                                                                         
 retcode : 0                                                                    
DEBUG 2015-05-18 23:38:19,268:   Executing :                                    
 command : voms-proxy-info -file /tmp/x509up_u112870 -actimeleft                
 output : 85755                                                                 

 error:                                                                         
 retcode : 0                                                                    
DEBUG 2015-05-18 23:38:19,268:   Proxy valid for 23:49:15 hours                 
INFO 2015-05-18 23:38:19,268:    Will use CRAB configuration file crab.py       
DEBUG 2015-05-18 23:38:19,268:   Loading CRAB configuration file.               
ERROR 2015-05-18 23:38:19,271:   Syntax error in CRAB configuration file:       
'NoneType' object has no attribute 'endswith'                                   
Please refer to https://twiki.cern.ch/twiki/bin/view/CMSPublic/CRAB3CommonErrors#Syntax_error_in_CRAB_configurati
See the ./crab.log file for more details.                                                 
The documentation about the CRAB configuration file can be found in https://twiki.cern.ch/twiki/bin/view/CMSPublic/CRAB3ConfigurationFile
ERROR 2015-05-18 23:38:19,271:   Caught exception                               
Traceback (most recent call last):                                              
  File "/gpfs22/home/meloam/projects/bsm-edntuple/me2/src/BSM3GAna/EDMNtuple/CRABClient/src/python/CRABClient/CRABCommandLine.py", line 139, in <module>
    client()                                                                    
  File "/gpfs22/home/meloam/projects/bsm-edntuple/me2/src/BSM3GAna/EDMNtuple/CRABClient/src/python/CRABClient/CRABCommandLine.py", line 124, in __call__
    self.cmd = sub_cmd(self.logger, args[1:])                                   
  File "/home/meloam/projects/bsm-edntuple/me2/src/BSM3GAna/EDMNtuple/CRABClient/src/python/CRABClient/Commands/submit.py", line 39, in __init__
    SubCommand.__init__(self, logger, cmdargs, disable_interspersed_args = True)
  File "/home/meloam/projects/bsm-edntuple/me2/src/BSM3GAna/EDMNtuple/CRABClient/src/python/CRABClient/Commands/SubCommand.py", line 290, in __init__
    self.loadConfig(self.options.config, self.args)                             
  File "/home/meloam/projects/bsm-edntuple/me2/src/BSM3GAna/EDMNtuple/CRABClient/src/python/CRABClient/Commands/SubCommand.py", line 83, in loadConfig
    raise ConfigurationException(configmsg)                                     
ConfigurationException: Syntax error in CRAB configuration file:                
'NoneType' object has no attribute 'endswith'                                   
Please refer to https://twiki.cern.ch/twiki/bin/view/CMSPublic/CRAB3CommonErrors#Syntax_error_in_CRAB_configurati
See the ./crab.log file for more details.                                       
The documentation about the CRAB configuration file can be found in https://twiki.cern.ch/twiki/bin/view/CMSPublic/CRAB3ConfigurationFile

Reraising a new exception blows away the context that would be needed to fix the problem. Who was set to NoneType??

PerilousApricot commented 9 years ago

Looking at it more, it's worse than it appears. Not only does the traceback for the real error get blown away, but it gets replaced by a pretty glaring bug in the exception handling code:

https://github.com/dmwm/CRABClient/blob/master/src/python/CRABClient/Commands/SubCommand.py#L91

    def _extractReason(self, configname, re):                                      
        """                                                                        
        To call in case of error loading the configuration file                    
        Get the reason of the failure without the stacktrace. Put the stacktrace in the crab.log file
        """                                                                        
        #get only the error wihtout the stacktrace                                 
        msg = str(re)                                                              
        filename = os.path.abspath( configname )                                   
        cfgBaseName = os.path.basename( filename ).replace(".py", "")              
        cfgDirName = os.path.dirname( filename )                                   
        if  not cfgDirName:                                                        
            modPath = imp.find_module(cfgBaseName)                                 
        else:                                                                      
            modPath = imp.find_module(cfgBaseName, [cfgDirName])                   
        try:                                                                       
            modRef = imp.load_module(cfgBaseName, modPath[0],                      
                                     modPath[1], modPath[2])                       
        except Exception, ex:                                                      
            msg = str(ex)                                                          

        return msg

The second try-except causes the error that's reported to be from imp.load_module, not from the one above. The whole function doesn't make sense though. Why spend all that time extracting out module information if it's not actually used?

mmascher commented 9 years ago

Hi Andrew,

I think this got broken at some point during a refactoring and nobody ever noticed. Thanks for reporting.

_extractReason is supposed to extract the reson so that we only print the traceback in a crab.log logfile. Honestly I have never liked that. I agree that stacktraces should not make it to users, but in this particular case it's a stacktrace provoked by the user's code, so he is interested in it.

belforte commented 8 years ago

@mmascher this is too technically tricky for me but maybe you can hand it over to @emaszs with some coaching

belforte commented 7 years ago

@mmascher @emaszs is this something we still have to address, or shall we close ?