riga / law

Build large-scale task workflows: luigi + job submission + remote targets + environment sandboxing using Docker/Singularity
http://law.readthedocs.io
BSD 3-Clause "New" or "Revised" License
96 stars 39 forks source link

Some classic workflows won't run #162

Closed lmoureaux closed 1 year ago

lmoureaux commented 1 year ago

Bug description

I have the following setup (copying only the relevant bits):

class RetrainAllShiftedSystematics(Task, law.LocalWorkflow):
    def create_branch_map(self):
        return {...}

class MultiInjectionClassifierTraining(RetrainAllShiftedSystematics):
    pass

The MultiInjectionClassifierTraining won't run with the following error being emitted:

WARNING: luigi-interface - Will not run MultiInjectionClassifierTraining(effective_workflow=local, branch=-1, downsample=100.0, signal=XToYYprimeTo4Q_MX3000_MY170_MYprime170_narrow_TuneCP5_13TeV-madgraph-pythia8_TIMBER.h5, lumi=137.59, 
xsec=3.7, efficiency=0.01, mass=-1, workflow=local) or any dependencies due to error in complete() method:
Traceback (most recent call last):
  File "/afs/desy.de/user/r/riegerma/public/law_sw/luigi_3/luigi/worker.py", line 434, in check_complete
    is_complete = check_complete_cached(task, completion_cache)
  File "/afs/desy.de/user/r/riegerma/public/law_sw/luigi_3/luigi/worker.py", line 419, in check_complete_cached
    is_complete = task.complete()
  File "/afs/desy.de/user/r/riegerma/public/law_sw/law/law/workflow/base.py", line 109, in complete
    return super(BaseWorkflowProxy, self).complete()
  File "/afs/desy.de/user/r/riegerma/public/law_sw/law/law/task/base.py", line 240, in complete
    outputs = flatten(self.output())
  File "/afs/desy.de/user/r/riegerma/public/law_sw/law/law/workflow/base.py", line 128, in output
    targets = luigi.task.getpaths(self.task.get_branch_tasks())
  File "/afs/desy.de/user/r/riegerma/public/law_sw/law/law/workflow/base.py", line 997, in get_branch_tasks
    for b in self.get_branch_map():
  File "/afs/desy.de/user/r/riegerma/public/law_sw/law/law/workflow/base.py", line 948, in get_branch_map
    if is_classmethod(self.create_branch_map, self.__class__):
  File "/afs/desy.de/user/r/riegerma/public/law_sw/law/law/util.py", line 260, in is_classmethod
    return cls.__dict__[func.__name__].__class__.__name__ == "classmethod"
KeyError: 'create_branch_map'

This is because create_branch_map is inherited from a parent class and thus not in __dict__.

riga commented 1 year ago

Thanks @lmoureaux for reporting! Going to fix this right away!

lmoureaux commented 1 year ago

Works, thanks a lot for your reactivity!