search

DataBiosphere / toil

A scalable, efficient, cross-platform (Linux/macOS) and easy-to-use workflow engine in pure Python.
http://toil.ucsc-cgl.org/.
Apache License 2.0
900 stars 240 forks source link

AttributeError: 'Expando' object has no attribute 'workers' #609

Closed scottfrazer closed 9 years ago

scottfrazer commented 9 years ago

When I run:

$ rm -rf toilWorkflow/; python src/toil/wdl/wdltoil.py

I get this:

INFO:toil.lib.bioio:Logging set at level: INFO
INFO:toil.common:Using the single machine batch system
INFO:toil.jobStores.fileJobStore:Jobstore directory is: /Users/sfrazer/projects/toil/toilWorkflow
WARNING:toil.batchSystems.singleMachine:Limiting maxCores to CPU count of system (8).
WARNING:toil.batchSystems.singleMachine:Limiting maxMemory to physically available memory (8589934592).
INFO:toil.batchSystems.singleMachine:Setting up the thread pool with 80 workers, given a minimum CPU fraction of 0.100000 and a maximum CPU value of 8.
INFO:toil.common:Written the environment for the jobs to the environment file
INFO:toil.leader:Checked batch system has no running jobs and no updated jobs
INFO:toil.leader:Found 1 jobs to start and 0 jobs with successors to run
INFO:toil.leader:Starting the main loop
INFO:toil.batchSystems.singleMachine:Executing command: '/Users/sfrazer/venv/wdl-toil/bin/_toil_worker /Users/sfrazer/projects/toil/toilWorkflow b/V/jobUulNcW'.
INFO:toil.batchSystems.singleMachine:Executing command: '/Users/sfrazer/venv/wdl-toil/bin/_toil_worker /Users/sfrazer/projects/toil/toilWorkflow X/J/joby_OmSG'.
INFO:toil.batchSystems.singleMachine:Executing command: '/Users/sfrazer/venv/wdl-toil/bin/_toil_worker /Users/sfrazer/projects/toil/toilWorkflow k/r/jobSMCrWI'.
INFO:toil.batchSystems.singleMachine:Executing command: '/Users/sfrazer/venv/wdl-toil/bin/_toil_worker /Users/sfrazer/projects/toil/toilWorkflow b/V/jobUulNcW'.
Process Process-1:
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/sfrazer/projects/toil/src/toil/leader.py", line 67, in statsAndLoggingAggregatorProcess
    if jobStore.readStatsAndLogging(callback) == 0:
  File "/Users/sfrazer/projects/toil/src/toil/jobStores/fileJobStore.py", line 238, in readStatsAndLogging
    callback(fH)
  File "/Users/sfrazer/projects/toil/src/toil/leader.py", line 47, in callback
    workers = stats.workers
AttributeError: 'Expando' object has no attribute 'workers'
INFO:toil.leader:Only failed jobs and their dependents (0 total) are remaining, so exiting.
INFO:toil.leader:Finished the main loop
INFO:toil.leader:Waiting for stats and logging collator process to finish
INFO:toil.leader:Stats/logging finished collating in 0.000979900360107 seconds
None

It was actually working for a little while and then suddenly it started giving back this error and I don't quite know why. Any ideas?

dgaston commented 9 years ago

I'm getting this error as well with toil version 3.1. It also seems as if now I am only running a single job at a time on my local machine when it should have enough resources allocated/requested to be running several

hannes-ucsc commented 9 years ago

As this is a bug in a release, we should fix this ASAP.

hannes-ucsc commented 9 years ago

The fix for this is included in the 3.1.1 release. Running pip install --upgrade toil will install it.

scottfrazer commented 9 years ago

Thanks!! On Fri, Dec 4, 2015 at 9:11 PM Hannes Schmidt notifications@github.com wrote:

The fix for this is included in the 3.1.1 release. Running pip install --upgrade toil will install it.

— Reply to this email directly or view it on GitHub https://github.com/BD2KGenomics/toil/issues/609#issuecomment-162131598.