search

radical-cybertools / radical.entk

The RADICAL Ensemble Toolkit
https://radical-cybertools.github.io/entk/index.html
Other
28 stars 17 forks source link

Node configuration is not available #293

Closed FreddieSun closed 5 years ago

FreddieSun commented 5 years ago

I met a problem while I submit the EnTK'job to Comet. 

2018-12-19 00:36:37,381: radical.saga.cpi    : pmgr.0000.launching.0           : Thread-2       : ERROR   : NoSuccess: Couldn't get job id from submitted job! sbatch output:
sbatch: error: Batch job submission failed: Requested node configuration is not available
removed `tmp_QY7fu7.slurm'

The full log is here

2018-12-19 00:36:16,267: radical.entk.appmanager.0000: MainProcess                     : MainThread     : INFO    : python.interpreter   version: 2.7.12 (default, Nov 12 2018, 14:36:49) [GCC 5.4.0 20160609]
2018-12-19 00:36:16,268: radical.entk.appmanager.0000: MainProcess                     : MainThread     : INFO    :                      pid/tid: 4634/MainThread
2018-12-19 00:36:16,268: radical.entk.appmanager.0000: MainProcess                     : MainThread     : INFO    : Application Manager initialized
2018-12-19 00:36:16,479: radical.entk.task_processor: MainProcess                     : MainThread     : INFO    : python.interpreter   version: 2.7.12 (default, Nov 12 2018, 14:36:49) [GCC 5.4.0 20160609]
2018-12-19 00:36:16,479: radical.entk.task_processor: MainProcess                     : MainThread     : INFO    :                      pid/tid: 4634/MainThread
2018-12-19 00:36:16,480: radical.entk.resource_manager.0000: MainProcess                     : MainThread     : INFO    : python.interpreter   version: 2.7.12 (default, Nov 12 2018, 14:36:49) [GCC 5.4.0 20160609]
2018-12-19 00:36:16,480: radical.entk.resource_manager.0000: MainProcess                     : MainThread     : INFO    :                      pid/tid: 4634/MainThread
2018-12-19 00:36:16,480: radical.entk.resource_manager.0000: MainProcess                     : MainThread     : INFO    : Created resource manager object: resource_manager.0000
2018-12-19 00:36:16,480: radical.entk.resource_manager.0000: MainProcess                     : MainThread     : INFO    : Resource description validated
2018-12-19 00:36:16,480: radical.entk.appmanager.0000: MainProcess                     : MainThread     : INFO    : Workflow assigned to Application Manager
2018-12-19 00:36:16,533: radical.entk.wfprocessor.0000: MainProcess                     : MainThread     : INFO    : python.interpreter   version: 2.7.12 (default, Nov 12 2018, 14:36:49) [GCC 5.4.0 20160609]
2018-12-19 00:36:16,533: radical.entk.wfprocessor.0000: MainProcess                     : MainThread     : INFO    :                      pid/tid: 4634/MainThread
2018-12-19 00:36:16,534: radical.entk.wfprocessor.0000: MainProcess                     : MainThread     : INFO    : Created WFProcessor object: wfprocessor.0000
2018-12-19 00:36:16,548: radical.entk.appmanager.0000: MainProcess                     : MainThread     : INFO    : Starting resource request submission
2018-12-19 00:36:27,305: radical.entk.resource_manager.0000: MainProcess                     : MainThread     : INFO    : Pilot pilot.0000 state: PMGR_LAUNCHING_PENDING
2018-12-19 00:36:27,307: radical.entk.resource_manager.0000: MainProcess                     : MainThread     : INFO    : Resource request submission successful.. waiting for pilot to go Active
2018-12-19 00:36:27,317: radical.entk.resource_manager.0000: MainProcess                     : pmgr.0000.subscriber._state_sub_cb: INFO    : Pilot pilot.0000 state: PMGR_LAUNCHING
2018-12-19 00:36:37,381: radical.saga.cpi    : pmgr.0000.launching.0           : Thread-2       : ERROR   : NoSuccess: Couldn't get job id from submitted job! sbatch output:
sbatch: error: Batch job submission failed: Requested node configuration is not available
removed `tmp_QY7fu7.slurm'

2018-12-19 00:36:37,387: radical.entk.resource_manager.0000: MainProcess                     : pmgr.0000.subscriber._state_sub_cb: INFO    : Pilot pilot.0000 state: FAILED
2018-12-19 00:36:37,388: radical.entk.resource_manager.0000: MainProcess                     : pmgr.0000.subscriber._state_sub_cb: ERROR   : Pilot has failed
2018-12-19 00:36:37,428: radical.entk.resource_manager.0000: MainProcess                     : MainThread     : INFO    : Pilot is now active
2018-12-19 00:36:37,429: radical.entk.appmanager.0000: MainProcess                     : MainThread     : INFO    : Starting synchronizer thread
2018-12-19 00:36:37,430: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : synchronizer thread started
2018-12-19 00:36:37,430: radical.entk.appmanager.0000: MainProcess                     : MainThread     : INFO    : Starting WFProcessor process from AppManager
2018-12-19 00:36:37,432: radical.entk.wfprocessor.0000: MainProcess                     : MainThread     : INFO    : Starting WFprocessor process
2018-12-19 00:36:37,444: radical.entk.task_manager.0000: MainProcess                     : MainThread     : INFO    : python.interpreter   version: 2.7.12 (default, Nov 12 2018, 14:36:49) [GCC 5.4.0 20160609]
2018-12-19 00:36:37,445: radical.entk.task_manager.0000: MainProcess                     : MainThread     : INFO    :                      pid/tid: 4634/MainThread
2018-12-19 00:36:37,452: radical.entk.wfprocessor.0000: wfprocessor                     : MainThread     : INFO    : WFprocessor started
2018-12-19 00:36:37,453: radical.entk.wfprocessor.0000: wfprocessor                     : MainThread     : INFO    : Starting dequeue-thread
2018-12-19 00:36:37,454: radical.entk.wfprocessor.0000: wfprocessor                     : dequeue-thread : INFO    : Dequeue thread started
2018-12-19 00:36:37,455: radical.entk.wfprocessor.0000: wfprocessor                     : MainThread     : INFO    : Starting enqueue-thread
2018-12-19 00:36:37,458: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : enqueue-thread started
2018-12-19 00:36:37,476: radical.entk.task_manager.0000: MainProcess                     : MainThread     : INFO    : Created task manager object: task_manager.0000
2018-12-19 00:36:37,476: radical.entk.appmanager.0000: MainProcess                     : MainThread     : INFO    : Starting task manager process from AppManager
2018-12-19 00:36:37,478: radical.entk.task_manager.0000: MainProcess                     : MainThread     : INFO    : Starting task manager process
2018-12-19 00:36:37,483: radical.entk.task_manager.0000: MainProcess                     : MainThread     : INFO    : Starting heartbeat thread
2018-12-19 00:36:37,489: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received pipeline.0000 with state SCHEDULING
2018-12-19 00:36:37,489: radical.entk.appmanager.0000: MainProcess                     : MainThread     : INFO    : Terminating WFprocessor
2018-12-19 00:36:37,489: radical.entk.task_manager.0000: task-manager                    : MainThread     : INFO    : Task Manager process started
2018-12-19 00:36:37,490: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Found pipeline pipeline.0000, state SCHEDULING, completed False
2018-12-19 00:36:37,491: radical.entk.wfprocessor.0000: wfprocessor                     : MainThread     : INFO    : Terminating enqueue-thread
2018-12-19 00:36:37,494: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of pipeline.0000 to new state SCHEDULING successful
2018-12-19 00:36:37,498: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received stage.0000 with state SCHEDULING
2018-12-19 00:36:37,499: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Found parent pipeline: pipeline.0000
2018-12-19 00:36:37,500: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of stage.0000 to new state SCHEDULING successful
2018-12-19 00:36:37,502: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0000 with state SCHEDULING
2018-12-19 00:36:37,505: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0000 to new state SCHEDULING successful
2018-12-19 00:36:37,507: radical.entk.task_manager.0000: MainProcess                     : heartbeat      : INFO    : Sent heartbeat request
2018-12-19 00:36:37,508: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0001 with state SCHEDULING
2018-12-19 00:36:37,509: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0001 to new state SCHEDULING successful
2018-12-19 00:36:37,512: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0002 with state SCHEDULING
2018-12-19 00:36:37,514: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0002 to new state SCHEDULING successful
2018-12-19 00:36:37,515: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0003 with state SCHEDULING
2018-12-19 00:36:37,517: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0003 to new state SCHEDULING successful
2018-12-19 00:36:37,519: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0004 with state SCHEDULING
2018-12-19 00:36:37,522: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0004 to new state SCHEDULING successful
2018-12-19 00:36:37,524: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0005 with state SCHEDULING
2018-12-19 00:36:37,525: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0005 to new state SCHEDULING successful
2018-12-19 00:36:37,527: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0006 with state SCHEDULING
2018-12-19 00:36:37,528: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0006 to new state SCHEDULING successful
2018-12-19 00:36:37,531: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0007 with state SCHEDULING
2018-12-19 00:36:37,532: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0007 to new state SCHEDULING successful
2018-12-19 00:36:37,534: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0008 with state SCHEDULING
2018-12-19 00:36:37,535: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0008 to new state SCHEDULING successful
2018-12-19 00:36:37,538: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0009 with state SCHEDULING
2018-12-19 00:36:37,539: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0009 to new state SCHEDULING successful
2018-12-19 00:36:37,541: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0010 with state SCHEDULING
2018-12-19 00:36:37,542: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0010 to new state SCHEDULING successful
2018-12-19 00:36:37,543: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0011 with state SCHEDULING
2018-12-19 00:36:37,545: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0011 to new state SCHEDULING successful
2018-12-19 00:36:37,547: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0012 with state SCHEDULING
2018-12-19 00:36:37,548: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0012 to new state SCHEDULING successful
2018-12-19 00:36:37,550: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0013 with state SCHEDULING
2018-12-19 00:36:37,551: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0013 to new state SCHEDULING successful
2018-12-19 00:36:37,553: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0014 with state SCHEDULING
2018-12-19 00:36:37,554: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0014 to new state SCHEDULING successful
2018-12-19 00:36:37,556: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0015 with state SCHEDULING
2018-12-19 00:36:37,557: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0015 to new state SCHEDULING successful
2018-12-19 00:36:37,558: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0016 with state SCHEDULING
2018-12-19 00:36:37,560: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0016 to new state SCHEDULING successful
2018-12-19 00:36:37,562: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0017 with state SCHEDULING
2018-12-19 00:36:37,563: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0017 to new state SCHEDULING successful
2018-12-19 00:36:37,564: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0018 with state SCHEDULING
2018-12-19 00:36:37,566: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0018 to new state SCHEDULING successful
2018-12-19 00:36:37,568: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0019 with state SCHEDULING
2018-12-19 00:36:37,569: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0019 to new state SCHEDULING successful
2018-12-19 00:36:37,570: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0020 with state SCHEDULING
2018-12-19 00:36:37,571: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0020 to new state SCHEDULING successful
2018-12-19 00:36:37,573: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0021 with state SCHEDULING
2018-12-19 00:36:37,574: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0021 to new state SCHEDULING successful
2018-12-19 00:36:37,576: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0022 with state SCHEDULING
2018-12-19 00:36:37,577: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0022 to new state SCHEDULING successful
2018-12-19 00:36:37,579: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0023 with state SCHEDULING
2018-12-19 00:36:37,580: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0023 to new state SCHEDULING successful
2018-12-19 00:36:37,581: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0024 with state SCHEDULING
2018-12-19 00:36:37,583: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0024 to new state SCHEDULING successful
2018-12-19 00:36:37,584: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0025 with state SCHEDULING
2018-12-19 00:36:37,585: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0025 to new state SCHEDULING successful
2018-12-19 00:36:37,587: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0026 with state SCHEDULING
2018-12-19 00:36:37,588: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0026 to new state SCHEDULING successful
2018-12-19 00:36:37,589: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0027 with state SCHEDULING
2018-12-19 00:36:37,590: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0027 to new state SCHEDULING successful
2018-12-19 00:36:37,592: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0028 with state SCHEDULING
2018-12-19 00:36:37,593: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0028 to new state SCHEDULING successful
2018-12-19 00:36:37,595: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0029 with state SCHEDULING
2018-12-19 00:36:37,596: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0029 to new state SCHEDULING successful
2018-12-19 00:36:37,597: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0030 with state SCHEDULING
2018-12-19 00:36:37,598: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0030 to new state SCHEDULING successful
2018-12-19 00:36:37,599: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0031 with state SCHEDULING
2018-12-19 00:36:37,601: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0031 to new state SCHEDULING successful
2018-12-19 00:36:37,601: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0032 with state SCHEDULING
2018-12-19 00:36:37,603: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0032 to new state SCHEDULING successful
2018-12-19 00:36:37,605: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0033 with state SCHEDULING
2018-12-19 00:36:37,606: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0033 to new state SCHEDULING successful
2018-12-19 00:36:37,607: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0034 with state SCHEDULING
2018-12-19 00:36:37,608: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0034 to new state SCHEDULING successful
2018-12-19 00:36:37,609: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0035 with state SCHEDULING
2018-12-19 00:36:37,610: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0035 to new state SCHEDULING successful
2018-12-19 00:36:37,612: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0036 with state SCHEDULING
2018-12-19 00:36:37,613: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0036 to new state SCHEDULING successful
2018-12-19 00:36:37,614: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0037 with state SCHEDULING
2018-12-19 00:36:37,615: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0037 to new state SCHEDULING successful
2018-12-19 00:36:37,617: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0038 with state SCHEDULING
2018-12-19 00:36:37,618: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0038 to new state SCHEDULING successful
2018-12-19 00:36:37,619: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0039 with state SCHEDULING
2018-12-19 00:36:37,620: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0039 to new state SCHEDULING successful
2018-12-19 00:36:37,621: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0040 with state SCHEDULING
2018-12-19 00:36:37,622: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0040 to new state SCHEDULING successful
2018-12-19 00:36:37,626: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0041 with state SCHEDULING
2018-12-19 00:36:37,628: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0041 to new state SCHEDULING successful
2018-12-19 00:36:37,631: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0042 with state SCHEDULING
2018-12-19 00:36:37,632: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0042 to new state SCHEDULING successful
2018-12-19 00:36:37,634: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0043 with state SCHEDULING
2018-12-19 00:36:37,637: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0043 to new state SCHEDULING successful
2018-12-19 00:36:37,638: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0044 with state SCHEDULING
2018-12-19 00:36:37,640: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0044 to new state SCHEDULING successful
2018-12-19 00:36:37,642: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0045 with state SCHEDULING
2018-12-19 00:36:37,644: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0045 to new state SCHEDULING successful
2018-12-19 00:36:37,647: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0046 with state SCHEDULING
2018-12-19 00:36:37,649: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0046 to new state SCHEDULING successful
2018-12-19 00:36:37,651: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0047 with state SCHEDULING
2018-12-19 00:36:37,652: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0047 to new state SCHEDULING successful
2018-12-19 00:36:37,654: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0048 with state SCHEDULING
2018-12-19 00:36:37,656: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0048 to new state SCHEDULING successful
2018-12-19 00:36:37,657: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0049 with state SCHEDULING
2018-12-19 00:36:37,659: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0049 to new state SCHEDULING successful
2018-12-19 00:36:37,661: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0050 with state SCHEDULING
2018-12-19 00:36:37,662: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0050 to new state SCHEDULING successful
2018-12-19 00:36:37,665: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0051 with state SCHEDULING
2018-12-19 00:36:37,667: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0051 to new state SCHEDULING successful
2018-12-19 00:36:37,668: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0052 with state SCHEDULING
2018-12-19 00:36:37,670: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0052 to new state SCHEDULING successful
2018-12-19 00:36:37,672: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0053 with state SCHEDULING
2018-12-19 00:36:37,673: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0053 to new state SCHEDULING successful
2018-12-19 00:36:37,676: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0054 with state SCHEDULING
2018-12-19 00:36:37,678: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0054 to new state SCHEDULING successful
2018-12-19 00:36:37,679: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0055 with state SCHEDULING
2018-12-19 00:36:37,680: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0055 to new state SCHEDULING successful
2018-12-19 00:36:37,681: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0056 with state SCHEDULING
2018-12-19 00:36:37,682: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0056 to new state SCHEDULING successful
2018-12-19 00:36:37,683: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0057 with state SCHEDULING
2018-12-19 00:36:37,684: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0057 to new state SCHEDULING successful
2018-12-19 00:36:37,685: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0058 with state SCHEDULING
2018-12-19 00:36:37,686: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0058 to new state SCHEDULING successful
2018-12-19 00:36:37,687: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0059 with state SCHEDULING
2018-12-19 00:36:37,688: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0059 to new state SCHEDULING successful
2018-12-19 00:36:37,689: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0060 with state SCHEDULING
2018-12-19 00:36:37,690: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0060 to new state SCHEDULING successful
2018-12-19 00:36:37,691: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0061 with state SCHEDULING
2018-12-19 00:36:37,692: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0061 to new state SCHEDULING successful
2018-12-19 00:36:37,693: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0062 with state SCHEDULING
2018-12-19 00:36:37,694: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0062 to new state SCHEDULING successful
2018-12-19 00:36:37,695: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0063 with state SCHEDULING
2018-12-19 00:36:37,696: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0063 to new state SCHEDULING successful
2018-12-19 00:36:37,697: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0064 with state SCHEDULING
2018-12-19 00:36:37,698: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0064 to new state SCHEDULING successful
2018-12-19 00:36:37,700: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0065 with state SCHEDULING
2018-12-19 00:36:37,701: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0065 to new state SCHEDULING successful
2018-12-19 00:36:37,702: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0066 with state SCHEDULING
2018-12-19 00:36:37,704: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0066 to new state SCHEDULING successful
2018-12-19 00:36:37,706: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0067 with state SCHEDULING
2018-12-19 00:36:37,707: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0067 to new state SCHEDULING successful
2018-12-19 00:36:37,708: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0068 with state SCHEDULING
2018-12-19 00:36:37,709: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0068 to new state SCHEDULING successful
2018-12-19 00:36:37,710: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0069 with state SCHEDULING
2018-12-19 00:36:37,712: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0069 to new state SCHEDULING successful
2018-12-19 00:36:37,712: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0070 with state SCHEDULING
2018-12-19 00:36:37,713: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0070 to new state SCHEDULING successful
2018-12-19 00:36:37,715: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0071 with state SCHEDULING
2018-12-19 00:36:37,716: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0071 to new state SCHEDULING successful
2018-12-19 00:36:37,717: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0072 with state SCHEDULING
2018-12-19 00:36:37,718: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0072 to new state SCHEDULING successful
2018-12-19 00:36:37,719: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0073 with state SCHEDULING
2018-12-19 00:36:37,720: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0073 to new state SCHEDULING successful
2018-12-19 00:36:37,721: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0074 with state SCHEDULING
2018-12-19 00:36:37,723: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0074 to new state SCHEDULING successful
2018-12-19 00:36:37,725: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0075 with state SCHEDULING
2018-12-19 00:36:37,726: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0075 to new state SCHEDULING successful
2018-12-19 00:36:37,727: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0076 with state SCHEDULING
2018-12-19 00:36:37,728: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0076 to new state SCHEDULING successful
2018-12-19 00:36:37,729: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0077 with state SCHEDULING
2018-12-19 00:36:37,730: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0077 to new state SCHEDULING successful
2018-12-19 00:36:37,732: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0078 with state SCHEDULING
2018-12-19 00:36:37,733: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0078 to new state SCHEDULING successful
2018-12-19 00:36:37,734: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0079 with state SCHEDULING
2018-12-19 00:36:37,735: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0079 to new state SCHEDULING successful
2018-12-19 00:36:37,737: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0080 with state SCHEDULING
2018-12-19 00:36:37,738: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0080 to new state SCHEDULING successful
2018-12-19 00:36:37,739: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0081 with state SCHEDULING
2018-12-19 00:36:37,740: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0081 to new state SCHEDULING successful
2018-12-19 00:36:37,741: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0082 with state SCHEDULING
2018-12-19 00:36:37,742: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0082 to new state SCHEDULING successful
2018-12-19 00:36:37,743: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0083 with state SCHEDULING
2018-12-19 00:36:37,744: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0083 to new state SCHEDULING successful
2018-12-19 00:36:37,745: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0084 with state SCHEDULING
2018-12-19 00:36:37,746: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0084 to new state SCHEDULING successful
2018-12-19 00:36:37,747: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0085 with state SCHEDULING
2018-12-19 00:36:37,748: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0085 to new state SCHEDULING successful
2018-12-19 00:36:37,750: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0086 with state SCHEDULING
2018-12-19 00:36:37,751: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0086 to new state SCHEDULING successful
2018-12-19 00:36:37,752: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0087 with state SCHEDULING
2018-12-19 00:36:37,753: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0087 to new state SCHEDULING successful
2018-12-19 00:36:37,754: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0088 with state SCHEDULING
2018-12-19 00:36:37,755: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0088 to new state SCHEDULING successful
2018-12-19 00:36:37,756: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0089 with state SCHEDULING
2018-12-19 00:36:37,758: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0089 to new state SCHEDULING successful
2018-12-19 00:36:37,761: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0090 with state SCHEDULING
2018-12-19 00:36:37,762: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0090 to new state SCHEDULING successful
2018-12-19 00:36:37,765: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0091 with state SCHEDULING
2018-12-19 00:36:37,766: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0091 to new state SCHEDULING successful
2018-12-19 00:36:37,769: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0092 with state SCHEDULING
2018-12-19 00:36:37,771: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0092 to new state SCHEDULING successful
2018-12-19 00:36:37,774: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0093 with state SCHEDULING
2018-12-19 00:36:37,776: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0093 to new state SCHEDULING successful
2018-12-19 00:36:37,778: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0094 with state SCHEDULING
2018-12-19 00:36:37,780: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0094 to new state SCHEDULING successful
2018-12-19 00:36:37,782: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0095 with state SCHEDULING
2018-12-19 00:36:37,784: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0095 to new state SCHEDULING successful
2018-12-19 00:36:37,787: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0096 with state SCHEDULING
2018-12-19 00:36:37,788: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0096 to new state SCHEDULING successful
2018-12-19 00:36:37,791: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0097 with state SCHEDULING
2018-12-19 00:36:37,793: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0097 to new state SCHEDULING successful
2018-12-19 00:36:37,795: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0098 with state SCHEDULING
2018-12-19 00:36:37,797: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0098 to new state SCHEDULING successful
2018-12-19 00:36:37,799: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0099 with state SCHEDULING
2018-12-19 00:36:37,801: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0099 to new state SCHEDULING successful
2018-12-19 00:36:37,804: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0100 with state SCHEDULING
2018-12-19 00:36:37,805: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0100 to new state SCHEDULING successful
2018-12-19 00:36:37,808: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0101 with state SCHEDULING
2018-12-19 00:36:37,809: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0101 to new state SCHEDULING successful
2018-12-19 00:36:37,812: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0102 with state SCHEDULING
2018-12-19 00:36:37,814: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0102 to new state SCHEDULING successful
2018-12-19 00:36:37,817: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0103 with state SCHEDULING
2018-12-19 00:36:37,818: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0103 to new state SCHEDULING successful
2018-12-19 00:36:37,820: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0104 with state SCHEDULING
2018-12-19 00:36:37,821: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0104 to new state SCHEDULING successful
2018-12-19 00:36:37,825: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0105 with state SCHEDULING
2018-12-19 00:36:37,826: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0105 to new state SCHEDULING successful
2018-12-19 00:36:37,829: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0106 with state SCHEDULING
2018-12-19 00:36:37,830: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0106 to new state SCHEDULING successful
2018-12-19 00:36:37,833: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0107 with state SCHEDULING
2018-12-19 00:36:37,835: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0107 to new state SCHEDULING successful
2018-12-19 00:36:37,837: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0108 with state SCHEDULING
2018-12-19 00:36:37,839: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0108 to new state SCHEDULING successful
2018-12-19 00:36:37,841: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0109 with state SCHEDULING
2018-12-19 00:36:37,843: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0109 to new state SCHEDULING successful
2018-12-19 00:36:37,845: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0110 with state SCHEDULING
2018-12-19 00:36:37,847: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0110 to new state SCHEDULING successful
2018-12-19 00:36:37,849: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0111 with state SCHEDULING
2018-12-19 00:36:37,851: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0111 to new state SCHEDULING successful
2018-12-19 00:36:37,852: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0112 with state SCHEDULING
2018-12-19 00:36:37,854: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0112 to new state SCHEDULING successful
2018-12-19 00:36:37,856: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0113 with state SCHEDULING
2018-12-19 00:36:37,858: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0113 to new state SCHEDULING successful
2018-12-19 00:36:37,861: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0114 with state SCHEDULING
2018-12-19 00:36:37,863: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0114 to new state SCHEDULING successful
2018-12-19 00:36:37,865: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0115 with state SCHEDULING
2018-12-19 00:36:37,867: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0115 to new state SCHEDULING successful
2018-12-19 00:36:37,869: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0116 with state SCHEDULING
2018-12-19 00:36:37,871: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0116 to new state SCHEDULING successful
2018-12-19 00:36:37,873: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0117 with state SCHEDULING
2018-12-19 00:36:37,874: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0117 to new state SCHEDULING successful
2018-12-19 00:36:37,876: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0118 with state SCHEDULING
2018-12-19 00:36:37,877: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0118 to new state SCHEDULING successful
2018-12-19 00:36:37,879: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0119 with state SCHEDULING
2018-12-19 00:36:37,881: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0119 to new state SCHEDULING successful
2018-12-19 00:36:37,882: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0120 with state SCHEDULING
2018-12-19 00:36:37,884: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0120 to new state SCHEDULING successful
2018-12-19 00:36:37,886: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0121 with state SCHEDULING
2018-12-19 00:36:37,888: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0121 to new state SCHEDULING successful
2018-12-19 00:36:37,890: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0122 with state SCHEDULING
2018-12-19 00:36:37,891: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0122 to new state SCHEDULING successful
2018-12-19 00:36:37,893: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0123 with state SCHEDULING
2018-12-19 00:36:37,895: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0123 to new state SCHEDULING successful
2018-12-19 00:36:37,898: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0124 with state SCHEDULING
2018-12-19 00:36:37,899: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0124 to new state SCHEDULING successful
2018-12-19 00:36:37,901: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0125 with state SCHEDULING
2018-12-19 00:36:37,903: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0125 to new state SCHEDULING successful
2018-12-19 00:36:37,904: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0126 with state SCHEDULING
2018-12-19 00:36:37,906: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0126 to new state SCHEDULING successful
2018-12-19 00:36:37,908: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0127 with state SCHEDULING
2018-12-19 00:36:37,909: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0127 to new state SCHEDULING successful
2018-12-19 00:36:37,911: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0128 with state SCHEDULING
2018-12-19 00:36:37,912: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0128 to new state SCHEDULING successful
2018-12-19 00:36:37,914: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0129 with state SCHEDULING
2018-12-19 00:36:37,916: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0129 to new state SCHEDULING successful
2018-12-19 00:36:37,918: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0130 with state SCHEDULING
2018-12-19 00:36:37,919: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0130 to new state SCHEDULING successful
2018-12-19 00:36:37,921: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0131 with state SCHEDULING
2018-12-19 00:36:37,923: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0131 to new state SCHEDULING successful
2018-12-19 00:36:37,925: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0132 with state SCHEDULING
2018-12-19 00:36:37,926: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0132 to new state SCHEDULING successful
2018-12-19 00:36:37,928: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0133 with state SCHEDULING
2018-12-19 00:36:37,930: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0133 to new state SCHEDULING successful
2018-12-19 00:36:37,932: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0134 with state SCHEDULING
2018-12-19 00:36:37,933: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0134 to new state SCHEDULING successful
2018-12-19 00:36:37,936: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0135 with state SCHEDULING
2018-12-19 00:36:37,938: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0135 to new state SCHEDULING successful
2018-12-19 00:36:37,941: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0136 with state SCHEDULING
2018-12-19 00:36:37,942: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0136 to new state SCHEDULING successful
2018-12-19 00:36:37,945: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0137 with state SCHEDULING
2018-12-19 00:36:37,946: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0137 to new state SCHEDULING successful
2018-12-19 00:36:37,949: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0138 with state SCHEDULING
2018-12-19 00:36:37,950: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0138 to new state SCHEDULING successful
2018-12-19 00:36:37,953: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0139 with state SCHEDULING
2018-12-19 00:36:37,955: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0139 to new state SCHEDULING successful
2018-12-19 00:36:37,958: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0140 with state SCHEDULING
2018-12-19 00:36:37,960: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0140 to new state SCHEDULING successful
2018-12-19 00:36:37,962: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0141 with state SCHEDULING
2018-12-19 00:36:37,964: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0141 to new state SCHEDULING successful
2018-12-19 00:36:37,965: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0142 with state SCHEDULING
2018-12-19 00:36:37,967: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0142 to new state SCHEDULING successful
2018-12-19 00:36:37,969: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0143 with state SCHEDULING
2018-12-19 00:36:37,970: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0143 to new state SCHEDULING successful
2018-12-19 00:36:37,972: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0144 with state SCHEDULING
2018-12-19 00:36:37,973: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0144 to new state SCHEDULING successful
2018-12-19 00:36:37,975: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0145 with state SCHEDULING
2018-12-19 00:36:37,977: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0145 to new state SCHEDULING successful
2018-12-19 00:36:37,979: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0146 with state SCHEDULING
2018-12-19 00:36:37,980: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0146 to new state SCHEDULING successful
2018-12-19 00:36:37,982: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0147 with state SCHEDULING
2018-12-19 00:36:37,983: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0147 to new state SCHEDULING successful
2018-12-19 00:36:37,986: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0148 with state SCHEDULING
2018-12-19 00:36:37,987: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0148 to new state SCHEDULING successful
2018-12-19 00:36:37,989: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0149 with state SCHEDULING
2018-12-19 00:36:37,990: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0149 to new state SCHEDULING successful
2018-12-19 00:36:37,992: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0150 with state SCHEDULING
2018-12-19 00:36:37,993: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0150 to new state SCHEDULING successful
2018-12-19 00:36:37,995: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0151 with state SCHEDULING
2018-12-19 00:36:37,997: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0151 to new state SCHEDULING successful
2018-12-19 00:36:37,999: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0152 with state SCHEDULING
2018-12-19 00:36:38,000: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0152 to new state SCHEDULING successful
2018-12-19 00:36:38,002: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0153 with state SCHEDULING
2018-12-19 00:36:38,003: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0153 to new state SCHEDULING successful
2018-12-19 00:36:38,005: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0154 with state SCHEDULING
2018-12-19 00:36:38,007: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0154 to new state SCHEDULING successful
2018-12-19 00:36:38,009: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0155 with state SCHEDULING
2018-12-19 00:36:38,010: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0155 to new state SCHEDULING successful
2018-12-19 00:36:38,012: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0156 with state SCHEDULING
2018-12-19 00:36:38,013: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0156 to new state SCHEDULING successful
2018-12-19 00:36:38,015: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0157 with state SCHEDULING
2018-12-19 00:36:38,016: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0157 to new state SCHEDULING successful
2018-12-19 00:36:38,018: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0158 with state SCHEDULING
2018-12-19 00:36:38,020: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0158 to new state SCHEDULING successful
2018-12-19 00:36:38,022: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0159 with state SCHEDULING
2018-12-19 00:36:38,022: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0159 to new state SCHEDULING successful
2018-12-19 00:36:38,024: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0160 with state SCHEDULING
2018-12-19 00:36:38,025: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0160 to new state SCHEDULING successful
2018-12-19 00:36:38,027: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0161 with state SCHEDULING
2018-12-19 00:36:38,028: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0161 to new state SCHEDULING successful
2018-12-19 00:36:38,030: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0162 with state SCHEDULING
2018-12-19 00:36:38,031: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0162 to new state SCHEDULING successful
2018-12-19 00:36:38,033: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0163 with state SCHEDULING
2018-12-19 00:36:38,034: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0163 to new state SCHEDULING successful
2018-12-19 00:36:38,037: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0164 with state SCHEDULING
2018-12-19 00:36:38,038: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0164 to new state SCHEDULING successful
2018-12-19 00:36:38,040: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0165 with state SCHEDULING
2018-12-19 00:36:38,041: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0165 to new state SCHEDULING successful
2018-12-19 00:36:38,043: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0166 with state SCHEDULING
2018-12-19 00:36:38,045: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0166 to new state SCHEDULING successful
2018-12-19 00:36:38,047: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0167 with state SCHEDULING
2018-12-19 00:36:38,048: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0167 to new state SCHEDULING successful
2018-12-19 00:36:38,051: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0168 with state SCHEDULING
2018-12-19 00:36:38,052: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0168 to new state SCHEDULING successful
2018-12-19 00:36:38,054: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0169 with state SCHEDULING
2018-12-19 00:36:38,056: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0169 to new state SCHEDULING successful
2018-12-19 00:36:38,058: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0170 with state SCHEDULING
2018-12-19 00:36:38,059: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0170 to new state SCHEDULING successful
2018-12-19 00:36:38,061: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0171 with state SCHEDULING
2018-12-19 00:36:38,062: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0171 to new state SCHEDULING successful
2018-12-19 00:36:38,065: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0172 with state SCHEDULING
2018-12-19 00:36:38,066: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0172 to new state SCHEDULING successful
2018-12-19 00:36:38,069: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0173 with state SCHEDULING
2018-12-19 00:36:38,070: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0173 to new state SCHEDULING successful
2018-12-19 00:36:38,073: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0174 with state SCHEDULING
2018-12-19 00:36:38,075: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0174 to new state SCHEDULING successful
2018-12-19 00:36:38,078: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0175 with state SCHEDULING
2018-12-19 00:36:38,079: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0175 to new state SCHEDULING successful
2018-12-19 00:36:38,082: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0176 with state SCHEDULING
2018-12-19 00:36:38,084: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0176 to new state SCHEDULING successful
2018-12-19 00:36:38,086: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0177 with state SCHEDULING
2018-12-19 00:36:38,088: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0177 to new state SCHEDULING successful
2018-12-19 00:36:38,091: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0178 with state SCHEDULING
2018-12-19 00:36:38,093: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0178 to new state SCHEDULING successful
2018-12-19 00:36:38,095: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0179 with state SCHEDULING
2018-12-19 00:36:38,096: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0179 to new state SCHEDULING successful
2018-12-19 00:36:38,099: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0180 with state SCHEDULING
2018-12-19 00:36:38,101: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0180 to new state SCHEDULING successful
2018-12-19 00:36:38,103: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0181 with state SCHEDULING
2018-12-19 00:36:38,104: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0181 to new state SCHEDULING successful
2018-12-19 00:36:38,105: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0182 with state SCHEDULING
2018-12-19 00:36:38,107: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0182 to new state SCHEDULING successful
2018-12-19 00:36:38,108: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0183 with state SCHEDULING
2018-12-19 00:36:38,110: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0183 to new state SCHEDULING successful
2018-12-19 00:36:38,113: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0184 with state SCHEDULING
2018-12-19 00:36:38,114: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0184 to new state SCHEDULING successful
2018-12-19 00:36:38,117: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0185 with state SCHEDULING
2018-12-19 00:36:38,119: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0185 to new state SCHEDULING successful
2018-12-19 00:36:38,120: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0186 with state SCHEDULING
2018-12-19 00:36:38,122: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0186 to new state SCHEDULING successful
2018-12-19 00:36:38,124: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0187 with state SCHEDULING
2018-12-19 00:36:38,127: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0187 to new state SCHEDULING successful
2018-12-19 00:36:38,128: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0188 with state SCHEDULING
2018-12-19 00:36:38,129: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0188 to new state SCHEDULING successful
2018-12-19 00:36:38,131: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0189 with state SCHEDULING
2018-12-19 00:36:38,133: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0189 to new state SCHEDULING successful
2018-12-19 00:36:38,135: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0190 with state SCHEDULING
2018-12-19 00:36:38,135: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0190 to new state SCHEDULING successful
2018-12-19 00:36:38,138: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0191 with state SCHEDULING
2018-12-19 00:36:38,139: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0191 to new state SCHEDULING successful
2018-12-19 00:36:38,146: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0000 with state SCHEDULED
2018-12-19 00:36:38,147: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0000 to new state SCHEDULED successful
2018-12-19 00:36:38,149: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0001 with state SCHEDULED
2018-12-19 00:36:38,150: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0001 to new state SCHEDULED successful
2018-12-19 00:36:38,152: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0002 with state SCHEDULED
2018-12-19 00:36:38,153: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0002 to new state SCHEDULED successful
2018-12-19 00:36:38,157: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0003 with state SCHEDULED
2018-12-19 00:36:38,158: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0003 to new state SCHEDULED successful
2018-12-19 00:36:38,159: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0004 with state SCHEDULED
2018-12-19 00:36:38,161: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0004 to new state SCHEDULED successful
2018-12-19 00:36:38,163: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0005 with state SCHEDULED
2018-12-19 00:36:38,164: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0005 to new state SCHEDULED successful
2018-12-19 00:36:38,165: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0006 with state SCHEDULED
2018-12-19 00:36:38,167: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0006 to new state SCHEDULED successful
2018-12-19 00:36:38,168: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0007 with state SCHEDULED
2018-12-19 00:36:38,169: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0007 to new state SCHEDULED successful
2018-12-19 00:36:38,171: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0008 with state SCHEDULED
2018-12-19 00:36:38,172: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0008 to new state SCHEDULED successful
2018-12-19 00:36:38,174: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0009 with state SCHEDULED
2018-12-19 00:36:38,175: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0009 to new state SCHEDULED successful
2018-12-19 00:36:38,178: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0010 with state SCHEDULED
2018-12-19 00:36:38,179: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0010 to new state SCHEDULED successful
2018-12-19 00:36:38,181: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0011 with state SCHEDULED
2018-12-19 00:36:38,182: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0011 to new state SCHEDULED successful
2018-12-19 00:36:38,184: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0012 with state SCHEDULED
2018-12-19 00:36:38,185: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0012 to new state SCHEDULED successful
2018-12-19 00:36:38,187: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0013 with state SCHEDULED
2018-12-19 00:36:38,188: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0013 to new state SCHEDULED successful
2018-12-19 00:36:38,189: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0014 with state SCHEDULED
2018-12-19 00:36:38,191: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0014 to new state SCHEDULED successful
2018-12-19 00:36:38,193: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0015 with state SCHEDULED
2018-12-19 00:36:38,194: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0015 to new state SCHEDULED successful
2018-12-19 00:36:38,196: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0016 with state SCHEDULED
2018-12-19 00:36:38,197: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0016 to new state SCHEDULED successful
2018-12-19 00:36:38,200: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0017 with state SCHEDULED
2018-12-19 00:36:38,201: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0017 to new state SCHEDULED successful
2018-12-19 00:36:38,203: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0018 with state SCHEDULED
2018-12-19 00:36:38,204: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0018 to new state SCHEDULED successful
2018-12-19 00:36:38,206: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0019 with state SCHEDULED
2018-12-19 00:36:38,207: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0019 to new state SCHEDULED successful
2018-12-19 00:36:38,209: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0020 with state SCHEDULED
2018-12-19 00:36:38,210: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0020 to new state SCHEDULED successful
2018-12-19 00:36:38,213: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0021 with state SCHEDULED
2018-12-19 00:36:38,214: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0021 to new state SCHEDULED successful
2018-12-19 00:36:38,216: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0022 with state SCHEDULED
2018-12-19 00:36:38,217: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0022 to new state SCHEDULED successful
2018-12-19 00:36:38,219: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0023 with state SCHEDULED
2018-12-19 00:36:38,221: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0023 to new state SCHEDULED successful
2018-12-19 00:36:38,223: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0024 with state SCHEDULED
2018-12-19 00:36:38,223: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0024 to new state SCHEDULED successful
2018-12-19 00:36:38,224: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0025 with state SCHEDULED
2018-12-19 00:36:38,226: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0025 to new state SCHEDULED successful
2018-12-19 00:36:38,228: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0026 with state SCHEDULED
2018-12-19 00:36:38,229: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0026 to new state SCHEDULED successful
2018-12-19 00:36:38,232: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0027 with state SCHEDULED
2018-12-19 00:36:38,233: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0027 to new state SCHEDULED successful
2018-12-19 00:36:38,235: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0028 with state SCHEDULED
2018-12-19 00:36:38,236: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0028 to new state SCHEDULED successful
2018-12-19 00:36:38,238: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0029 with state SCHEDULED
2018-12-19 00:36:38,239: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0029 to new state SCHEDULED successful
2018-12-19 00:36:38,242: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0030 with state SCHEDULED
2018-12-19 00:36:38,242: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0030 to new state SCHEDULED successful
2018-12-19 00:36:38,245: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0031 with state SCHEDULED
2018-12-19 00:36:38,246: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0031 to new state SCHEDULED successful
2018-12-19 00:36:38,248: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0032 with state SCHEDULED
2018-12-19 00:36:38,249: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0032 to new state SCHEDULED successful
2018-12-19 00:36:38,250: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0033 with state SCHEDULED
2018-12-19 00:36:38,251: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0033 to new state SCHEDULED successful
2018-12-19 00:36:38,253: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0034 with state SCHEDULED
2018-12-19 00:36:38,254: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0034 to new state SCHEDULED successful
2018-12-19 00:36:38,256: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0035 with state SCHEDULED
2018-12-19 00:36:38,257: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0035 to new state SCHEDULED successful
2018-12-19 00:36:38,259: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0036 with state SCHEDULED
2018-12-19 00:36:38,260: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0036 to new state SCHEDULED successful
2018-12-19 00:36:38,262: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0037 with state SCHEDULED
2018-12-19 00:36:38,264: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0037 to new state SCHEDULED successful
2018-12-19 00:36:38,266: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0038 with state SCHEDULED
2018-12-19 00:36:38,267: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0038 to new state SCHEDULED successful
2018-12-19 00:36:38,269: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0039 with state SCHEDULED
2018-12-19 00:36:38,270: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0039 to new state SCHEDULED successful
2018-12-19 00:36:38,272: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0040 with state SCHEDULED
2018-12-19 00:36:38,274: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0040 to new state SCHEDULED successful
2018-12-19 00:36:38,275: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0041 with state SCHEDULED
2018-12-19 00:36:38,277: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0041 to new state SCHEDULED successful
2018-12-19 00:36:38,279: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0042 with state SCHEDULED
2018-12-19 00:36:38,280: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0042 to new state SCHEDULED successful
2018-12-19 00:36:38,282: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0043 with state SCHEDULED
2018-12-19 00:36:38,283: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0043 to new state SCHEDULED successful
2018-12-19 00:36:38,285: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0044 with state SCHEDULED
2018-12-19 00:36:38,287: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0044 to new state SCHEDULED successful
2018-12-19 00:36:38,289: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0045 with state SCHEDULED
2018-12-19 00:36:38,290: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0045 to new state SCHEDULED successful
2018-12-19 00:36:38,292: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0046 with state SCHEDULED
2018-12-19 00:36:38,294: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0046 to new state SCHEDULED successful
2018-12-19 00:36:38,296: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0047 with state SCHEDULED
2018-12-19 00:36:38,297: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0047 to new state SCHEDULED successful
2018-12-19 00:36:38,299: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0048 with state SCHEDULED
2018-12-19 00:36:38,300: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0048 to new state SCHEDULED successful
2018-12-19 00:36:38,302: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0049 with state SCHEDULED
2018-12-19 00:36:38,303: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0049 to new state SCHEDULED successful
2018-12-19 00:36:38,305: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0050 with state SCHEDULED
2018-12-19 00:36:38,306: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0050 to new state SCHEDULED successful
2018-12-19 00:36:38,308: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0051 with state SCHEDULED
2018-12-19 00:36:38,309: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0051 to new state SCHEDULED successful
2018-12-19 00:36:38,311: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0052 with state SCHEDULED
2018-12-19 00:36:38,312: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0052 to new state SCHEDULED successful
2018-12-19 00:36:38,315: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0053 with state SCHEDULED
2018-12-19 00:36:38,316: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0053 to new state SCHEDULED successful
2018-12-19 00:36:38,318: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0054 with state SCHEDULED
2018-12-19 00:36:38,319: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0054 to new state SCHEDULED successful
2018-12-19 00:36:38,321: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0055 with state SCHEDULED
2018-12-19 00:36:38,322: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0055 to new state SCHEDULED successful
2018-12-19 00:36:38,324: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0056 with state SCHEDULED
2018-12-19 00:36:38,325: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0056 to new state SCHEDULED successful
2018-12-19 00:36:38,326: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0057 with state SCHEDULED
2018-12-19 00:36:38,327: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0057 to new state SCHEDULED successful
2018-12-19 00:36:38,330: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0058 with state SCHEDULED
2018-12-19 00:36:38,331: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0058 to new state SCHEDULED successful
2018-12-19 00:36:38,333: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0059 with state SCHEDULED
2018-12-19 00:36:38,334: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0059 to new state SCHEDULED successful
2018-12-19 00:36:38,336: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0060 with state SCHEDULED
2018-12-19 00:36:38,337: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0060 to new state SCHEDULED successful
2018-12-19 00:36:38,339: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0061 with state SCHEDULED
2018-12-19 00:36:38,340: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0061 to new state SCHEDULED successful
2018-12-19 00:36:38,343: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0062 with state SCHEDULED
2018-12-19 00:36:38,344: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0062 to new state SCHEDULED successful
2018-12-19 00:36:38,345: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0063 with state SCHEDULED
2018-12-19 00:36:38,347: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0063 to new state SCHEDULED successful
2018-12-19 00:36:38,349: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0064 with state SCHEDULED
2018-12-19 00:36:38,350: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0064 to new state SCHEDULED successful
2018-12-19 00:36:38,351: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0065 with state SCHEDULED
2018-12-19 00:36:38,352: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0065 to new state SCHEDULED successful
2018-12-19 00:36:38,354: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0066 with state SCHEDULED
2018-12-19 00:36:38,355: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0066 to new state SCHEDULED successful
2018-12-19 00:36:38,357: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0067 with state SCHEDULED
2018-12-19 00:36:38,358: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0067 to new state SCHEDULED successful
2018-12-19 00:36:38,361: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0068 with state SCHEDULED
2018-12-19 00:36:38,361: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0068 to new state SCHEDULED successful
2018-12-19 00:36:38,364: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0069 with state SCHEDULED
2018-12-19 00:36:38,365: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0069 to new state SCHEDULED successful
2018-12-19 00:36:38,367: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0070 with state SCHEDULED
2018-12-19 00:36:38,368: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0070 to new state SCHEDULED successful
2018-12-19 00:36:38,370: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0071 with state SCHEDULED
2018-12-19 00:36:38,371: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0071 to new state SCHEDULED successful
2018-12-19 00:36:38,372: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0072 with state SCHEDULED
2018-12-19 00:36:38,374: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0072 to new state SCHEDULED successful
2018-12-19 00:36:38,376: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0073 with state SCHEDULED
2018-12-19 00:36:38,378: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0073 to new state SCHEDULED successful
2018-12-19 00:36:38,380: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0074 with state SCHEDULED
2018-12-19 00:36:38,381: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0074 to new state SCHEDULED successful
2018-12-19 00:36:38,383: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0075 with state SCHEDULED
2018-12-19 00:36:38,384: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0075 to new state SCHEDULED successful
2018-12-19 00:36:38,387: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0076 with state SCHEDULED
2018-12-19 00:36:38,388: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0076 to new state SCHEDULED successful
2018-12-19 00:36:38,389: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0077 with state SCHEDULED
2018-12-19 00:36:38,391: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0077 to new state SCHEDULED successful
2018-12-19 00:36:38,393: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0078 with state SCHEDULED
2018-12-19 00:36:38,395: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0078 to new state SCHEDULED successful
2018-12-19 00:36:38,397: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0079 with state SCHEDULED
2018-12-19 00:36:38,398: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0079 to new state SCHEDULED successful
2018-12-19 00:36:38,400: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0080 with state SCHEDULED
2018-12-19 00:36:38,401: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0080 to new state SCHEDULED successful
2018-12-19 00:36:38,403: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0081 with state SCHEDULED
2018-12-19 00:36:38,404: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0081 to new state SCHEDULED successful
2018-12-19 00:36:38,406: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0082 with state SCHEDULED
2018-12-19 00:36:38,407: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0082 to new state SCHEDULED successful
2018-12-19 00:36:38,409: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0083 with state SCHEDULED
2018-12-19 00:36:38,411: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0083 to new state SCHEDULED successful
2018-12-19 00:36:38,412: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0084 with state SCHEDULED
2018-12-19 00:36:38,414: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0084 to new state SCHEDULED successful
2018-12-19 00:36:38,416: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0085 with state SCHEDULED
2018-12-19 00:36:38,418: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0085 to new state SCHEDULED successful
2018-12-19 00:36:38,419: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0086 with state SCHEDULED
2018-12-19 00:36:38,420: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0086 to new state SCHEDULED successful
2018-12-19 00:36:38,423: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0087 with state SCHEDULED
2018-12-19 00:36:38,424: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0087 to new state SCHEDULED successful
2018-12-19 00:36:38,426: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0088 with state SCHEDULED
2018-12-19 00:36:38,427: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0088 to new state SCHEDULED successful
2018-12-19 00:36:38,429: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0089 with state SCHEDULED
2018-12-19 00:36:38,430: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0089 to new state SCHEDULED successful
2018-12-19 00:36:38,432: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0090 with state SCHEDULED
2018-12-19 00:36:38,433: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0090 to new state SCHEDULED successful
2018-12-19 00:36:38,436: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0091 with state SCHEDULED
2018-12-19 00:36:38,437: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0091 to new state SCHEDULED successful
2018-12-19 00:36:38,439: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0092 with state SCHEDULED
2018-12-19 00:36:38,440: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0092 to new state SCHEDULED successful
2018-12-19 00:36:38,442: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0093 with state SCHEDULED
2018-12-19 00:36:38,443: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0093 to new state SCHEDULED successful
2018-12-19 00:36:38,446: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0094 with state SCHEDULED
2018-12-19 00:36:38,447: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0094 to new state SCHEDULED successful
2018-12-19 00:36:38,449: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0095 with state SCHEDULED
2018-12-19 00:36:38,450: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0095 to new state SCHEDULED successful
2018-12-19 00:36:38,452: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0096 with state SCHEDULED
2018-12-19 00:36:38,453: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0096 to new state SCHEDULED successful
2018-12-19 00:36:38,455: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0097 with state SCHEDULED
2018-12-19 00:36:38,456: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0097 to new state SCHEDULED successful
2018-12-19 00:36:38,458: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0098 with state SCHEDULED
2018-12-19 00:36:38,460: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0098 to new state SCHEDULED successful
2018-12-19 00:36:38,463: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0099 with state SCHEDULED
2018-12-19 00:36:38,464: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0099 to new state SCHEDULED successful
2018-12-19 00:36:38,466: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0100 with state SCHEDULED
2018-12-19 00:36:38,467: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0100 to new state SCHEDULED successful
2018-12-19 00:36:38,469: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0101 with state SCHEDULED
2018-12-19 00:36:38,471: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0101 to new state SCHEDULED successful
2018-12-19 00:36:38,473: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0102 with state SCHEDULED
2018-12-19 00:36:38,474: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0102 to new state SCHEDULED successful
2018-12-19 00:36:38,475: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0103 with state SCHEDULED
2018-12-19 00:36:38,477: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0103 to new state SCHEDULED successful
2018-12-19 00:36:38,479: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0104 with state SCHEDULED
2018-12-19 00:36:38,480: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0104 to new state SCHEDULED successful
2018-12-19 00:36:38,482: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0105 with state SCHEDULED
2018-12-19 00:36:38,483: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0105 to new state SCHEDULED successful
2018-12-19 00:36:38,485: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0106 with state SCHEDULED
2018-12-19 00:36:38,486: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0106 to new state SCHEDULED successful
2018-12-19 00:36:38,488: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0107 with state SCHEDULED
2018-12-19 00:36:38,489: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0107 to new state SCHEDULED successful
2018-12-19 00:36:38,491: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0108 with state SCHEDULED
2018-12-19 00:36:38,492: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0108 to new state SCHEDULED successful
2018-12-19 00:36:38,494: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0109 with state SCHEDULED
2018-12-19 00:36:38,496: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0109 to new state SCHEDULED successful
2018-12-19 00:36:38,498: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0110 with state SCHEDULED
2018-12-19 00:36:38,499: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0110 to new state SCHEDULED successful
2018-12-19 00:36:38,501: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0111 with state SCHEDULED
2018-12-19 00:36:38,502: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0111 to new state SCHEDULED successful
2018-12-19 00:36:38,504: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0112 with state SCHEDULED
2018-12-19 00:36:38,506: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0112 to new state SCHEDULED successful
2018-12-19 00:36:38,507: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0113 with state SCHEDULED
2018-12-19 00:36:38,509: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0113 to new state SCHEDULED successful
2018-12-19 00:36:38,511: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0114 with state SCHEDULED
2018-12-19 00:36:38,513: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0114 to new state SCHEDULED successful
2018-12-19 00:36:38,515: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0115 with state SCHEDULED
2018-12-19 00:36:38,516: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0115 to new state SCHEDULED successful
2018-12-19 00:36:38,518: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0116 with state SCHEDULED
2018-12-19 00:36:38,519: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0116 to new state SCHEDULED successful
2018-12-19 00:36:38,521: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0117 with state SCHEDULED
2018-12-19 00:36:38,523: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0117 to new state SCHEDULED successful
2018-12-19 00:36:38,524: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0118 with state SCHEDULED
2018-12-19 00:36:38,526: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0118 to new state SCHEDULED successful
2018-12-19 00:36:38,527: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0119 with state SCHEDULED
2018-12-19 00:36:38,529: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0119 to new state SCHEDULED successful
2018-12-19 00:36:38,531: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0120 with state SCHEDULED
2018-12-19 00:36:38,532: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0120 to new state SCHEDULED successful
2018-12-19 00:36:38,534: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0121 with state SCHEDULED
2018-12-19 00:36:38,535: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0121 to new state SCHEDULED successful
2018-12-19 00:36:38,537: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0122 with state SCHEDULED
2018-12-19 00:36:38,538: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0122 to new state SCHEDULED successful
2018-12-19 00:36:38,540: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0123 with state SCHEDULED
2018-12-19 00:36:38,541: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0123 to new state SCHEDULED successful
2018-12-19 00:36:38,543: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0124 with state SCHEDULED
2018-12-19 00:36:38,544: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0124 to new state SCHEDULED successful
2018-12-19 00:36:38,546: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0125 with state SCHEDULED
2018-12-19 00:36:38,547: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0125 to new state SCHEDULED successful
2018-12-19 00:36:38,549: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0126 with state SCHEDULED
2018-12-19 00:36:38,550: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0126 to new state SCHEDULED successful
2018-12-19 00:36:38,552: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0127 with state SCHEDULED
2018-12-19 00:36:38,553: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0127 to new state SCHEDULED successful
2018-12-19 00:36:38,561: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0128 with state SCHEDULED
2018-12-19 00:36:38,562: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0128 to new state SCHEDULED successful
2018-12-19 00:36:38,564: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0129 with state SCHEDULED
2018-12-19 00:36:38,566: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0129 to new state SCHEDULED successful
2018-12-19 00:36:38,567: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0130 with state SCHEDULED
2018-12-19 00:36:38,569: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0130 to new state SCHEDULED successful
2018-12-19 00:36:38,570: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0131 with state SCHEDULED
2018-12-19 00:36:38,572: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0131 to new state SCHEDULED successful
2018-12-19 00:36:38,575: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0132 with state SCHEDULED
2018-12-19 00:36:38,576: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0132 to new state SCHEDULED successful
2018-12-19 00:36:38,578: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0133 with state SCHEDULED
2018-12-19 00:36:38,579: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0133 to new state SCHEDULED successful
2018-12-19 00:36:38,582: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0134 with state SCHEDULED
2018-12-19 00:36:38,582: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0134 to new state SCHEDULED successful
2018-12-19 00:36:38,584: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0135 with state SCHEDULED
2018-12-19 00:36:38,585: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0135 to new state SCHEDULED successful
2018-12-19 00:36:38,587: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0136 with state SCHEDULED
2018-12-19 00:36:38,588: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0136 to new state SCHEDULED successful
2018-12-19 00:36:38,590: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0137 with state SCHEDULED
2018-12-19 00:36:38,592: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0137 to new state SCHEDULED successful
2018-12-19 00:36:38,594: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0138 with state SCHEDULED
2018-12-19 00:36:38,595: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0138 to new state SCHEDULED successful
2018-12-19 00:36:38,598: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0139 with state SCHEDULED
2018-12-19 00:36:38,599: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0139 to new state SCHEDULED successful
2018-12-19 00:36:38,602: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0140 with state SCHEDULED
2018-12-19 00:36:38,603: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0140 to new state SCHEDULED successful
2018-12-19 00:36:38,606: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0141 with state SCHEDULED
2018-12-19 00:36:38,608: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0141 to new state SCHEDULED successful
2018-12-19 00:36:38,610: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0142 with state SCHEDULED
2018-12-19 00:36:38,611: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0142 to new state SCHEDULED successful
2018-12-19 00:36:38,613: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0143 with state SCHEDULED
2018-12-19 00:36:38,615: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0143 to new state SCHEDULED successful
2018-12-19 00:36:38,617: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0144 with state SCHEDULED
2018-12-19 00:36:38,619: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0144 to new state SCHEDULED successful
2018-12-19 00:36:38,620: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0145 with state SCHEDULED
2018-12-19 00:36:38,622: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0145 to new state SCHEDULED successful
2018-12-19 00:36:38,624: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0146 with state SCHEDULED
2018-12-19 00:36:38,625: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0146 to new state SCHEDULED successful
2018-12-19 00:36:38,628: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0147 with state SCHEDULED
2018-12-19 00:36:38,629: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0147 to new state SCHEDULED successful
2018-12-19 00:36:38,632: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0148 with state SCHEDULED
2018-12-19 00:36:38,634: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0148 to new state SCHEDULED successful
2018-12-19 00:36:38,636: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0149 with state SCHEDULED
2018-12-19 00:36:38,637: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0149 to new state SCHEDULED successful
2018-12-19 00:36:38,640: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0150 with state SCHEDULED
2018-12-19 00:36:38,642: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0150 to new state SCHEDULED successful
2018-12-19 00:36:38,644: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0151 with state SCHEDULED
2018-12-19 00:36:38,645: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0151 to new state SCHEDULED successful
2018-12-19 00:36:38,648: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0152 with state SCHEDULED
2018-12-19 00:36:38,650: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0152 to new state SCHEDULED successful
2018-12-19 00:36:38,651: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0153 with state SCHEDULED
2018-12-19 00:36:38,653: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0153 to new state SCHEDULED successful
2018-12-19 00:36:38,655: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0154 with state SCHEDULED
2018-12-19 00:36:38,657: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0154 to new state SCHEDULED successful
2018-12-19 00:36:38,660: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0155 with state SCHEDULED
2018-12-19 00:36:38,661: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0155 to new state SCHEDULED successful
2018-12-19 00:36:38,662: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0156 with state SCHEDULED
2018-12-19 00:36:38,664: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0156 to new state SCHEDULED successful
2018-12-19 00:36:38,665: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0157 with state SCHEDULED
2018-12-19 00:36:38,666: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0157 to new state SCHEDULED successful
2018-12-19 00:36:38,669: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0158 with state SCHEDULED
2018-12-19 00:36:38,671: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0158 to new state SCHEDULED successful
2018-12-19 00:36:38,673: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0159 with state SCHEDULED
2018-12-19 00:36:38,675: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0159 to new state SCHEDULED successful
2018-12-19 00:36:38,677: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0160 with state SCHEDULED
2018-12-19 00:36:38,679: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0160 to new state SCHEDULED successful
2018-12-19 00:36:38,681: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0161 with state SCHEDULED
2018-12-19 00:36:38,683: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0161 to new state SCHEDULED successful
2018-12-19 00:36:38,685: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0162 with state SCHEDULED
2018-12-19 00:36:38,687: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0162 to new state SCHEDULED successful
2018-12-19 00:36:38,689: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0163 with state SCHEDULED
2018-12-19 00:36:38,691: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0163 to new state SCHEDULED successful
2018-12-19 00:36:38,694: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0164 with state SCHEDULED
2018-12-19 00:36:38,695: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0164 to new state SCHEDULED successful
2018-12-19 00:36:38,698: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0165 with state SCHEDULED
2018-12-19 00:36:38,699: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0165 to new state SCHEDULED successful
2018-12-19 00:36:38,701: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0166 with state SCHEDULED
2018-12-19 00:36:38,702: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0166 to new state SCHEDULED successful
2018-12-19 00:36:38,705: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0167 with state SCHEDULED
2018-12-19 00:36:38,706: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0167 to new state SCHEDULED successful
2018-12-19 00:36:38,709: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0168 with state SCHEDULED
2018-12-19 00:36:38,711: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0168 to new state SCHEDULED successful
2018-12-19 00:36:38,714: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0169 with state SCHEDULED
2018-12-19 00:36:38,715: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0169 to new state SCHEDULED successful
2018-12-19 00:36:38,717: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0170 with state SCHEDULED
2018-12-19 00:36:38,718: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0170 to new state SCHEDULED successful
2018-12-19 00:36:38,720: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0171 with state SCHEDULED
2018-12-19 00:36:38,721: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0171 to new state SCHEDULED successful
2018-12-19 00:36:38,723: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0172 with state SCHEDULED
2018-12-19 00:36:38,724: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0172 to new state SCHEDULED successful
2018-12-19 00:36:38,726: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0173 with state SCHEDULED
2018-12-19 00:36:38,727: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0173 to new state SCHEDULED successful
2018-12-19 00:36:38,729: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0174 with state SCHEDULED
2018-12-19 00:36:38,731: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0174 to new state SCHEDULED successful
2018-12-19 00:36:38,733: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0175 with state SCHEDULED
2018-12-19 00:36:38,734: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0175 to new state SCHEDULED successful
2018-12-19 00:36:38,736: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0176 with state SCHEDULED
2018-12-19 00:36:38,738: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0176 to new state SCHEDULED successful
2018-12-19 00:36:38,739: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0177 with state SCHEDULED
2018-12-19 00:36:38,741: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0177 to new state SCHEDULED successful
2018-12-19 00:36:38,743: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0178 with state SCHEDULED
2018-12-19 00:36:38,744: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0178 to new state SCHEDULED successful
2018-12-19 00:36:38,745: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0179 with state SCHEDULED
2018-12-19 00:36:38,747: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0179 to new state SCHEDULED successful
2018-12-19 00:36:38,749: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0180 with state SCHEDULED
2018-12-19 00:36:38,750: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0180 to new state SCHEDULED successful
2018-12-19 00:36:38,752: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0181 with state SCHEDULED
2018-12-19 00:36:38,753: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0181 to new state SCHEDULED successful
2018-12-19 00:36:38,755: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0182 with state SCHEDULED
2018-12-19 00:36:38,757: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0182 to new state SCHEDULED successful
2018-12-19 00:36:38,759: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0183 with state SCHEDULED
2018-12-19 00:36:38,760: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0183 to new state SCHEDULED successful
2018-12-19 00:36:38,762: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0184 with state SCHEDULED
2018-12-19 00:36:38,763: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0184 to new state SCHEDULED successful
2018-12-19 00:36:38,765: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0185 with state SCHEDULED
2018-12-19 00:36:38,767: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0185 to new state SCHEDULED successful
2018-12-19 00:36:38,768: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0186 with state SCHEDULED
2018-12-19 00:36:38,770: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0186 to new state SCHEDULED successful
2018-12-19 00:36:38,772: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0187 with state SCHEDULED
2018-12-19 00:36:38,773: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0187 to new state SCHEDULED successful
2018-12-19 00:36:38,775: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0188 with state SCHEDULED
2018-12-19 00:36:38,776: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0188 to new state SCHEDULED successful
2018-12-19 00:36:38,778: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0189 with state SCHEDULED
2018-12-19 00:36:38,780: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0189 to new state SCHEDULED successful
2018-12-19 00:36:38,782: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0190 with state SCHEDULED
2018-12-19 00:36:38,783: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0190 to new state SCHEDULED successful
2018-12-19 00:36:38,785: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received task.0191 with state SCHEDULED
2018-12-19 00:36:38,786: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of task.0191 to new state SCHEDULED successful
2018-12-19 00:36:38,789: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Received stage.0000 with state SCHEDULED
2018-12-19 00:36:38,789: radical.entk.appmanager.0000: MainProcess                     : synchronizer-thread: INFO    : Found parent pipeline: pipeline.0000
2018-12-19 00:36:38,790: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Transition of stage.0000 to new state SCHEDULED successful
2018-12-19 00:36:38,790: radical.entk.wfprocessor.0000: wfprocessor                     : enqueue-thread : INFO    : Enqueue thread terminated
2018-12-19 00:36:38,794: radical.entk.wfprocessor.0000: wfprocessor                     : MainThread     : INFO    : Terminating dequeue-thread
2018-12-19 00:36:38,795: radical.entk.wfprocessor.0000: wfprocessor                     : dequeue-thread : INFO    : Terminated dequeue thread
2018-12-19 00:36:38,798: radical.entk.appmanager.0000: MainProcess                     : MainThread     : INFO    : Terminating synchronizer thread
2018-12-19 00:36:38,799: radical.entk.appmanager.0000: MainProcess                     : MainThread     : INFO    : Synchronizer thread terminated
2018-12-19 00:36:38,799: radical.entk.appmanager.0000: MainProcess                     : MainThread     : INFO    : Terminating task manager process
/home/sun95/env/local/lib/python2.7/site-packages/pymongo/topology.py:149: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: http://api.mongodb.org/python/current/faq.html#is-pymongo-fork-safe
  "MongoClient opened before fork. Create MongoClient only "
2018-12-19 00:37:02,042: radical.entk.task_manager.0000: MainProcess                     : MainThread     : INFO    : Task manager process closed
2018-12-19 00:37:07,511: radical.entk.task_manager.0000: MainProcess                     : MainThread     : INFO    : Hearbeat thread terminated

Here I am measuring the strong scaling of EnTK. So I fix the task's size to be 192, and change the number of cpu needed. I changed it from 24, 48, 96, 192. For 24, 48, 96, they all work well. However, after i set the number of CPU needed to 192, I got the error.

Here is the code I submit: 

from radical.entk import Pipeline, Stage, Task, AppManager
import os
import random

# ------------------------------------------------------------------------------
# Set default verbosity

if os.environ.get('RADICAL_ENTK_VERBOSE') == None:
    os.environ['RADICAL_ENTK_REPORT'] = 'True'

# Description of how the RabbitMQ process is accessible
# No need to change/set any variables if you installed RabbitMQ has a system
# process. If you are running RabbitMQ under a docker container or another
# VM, set "RMQ_HOSTNAME" and "RMQ_PORT" in the session where you are running
# this script.
hostname = os.environ.get('RMQ_HOSTNAME', 'localhost')
port = os.environ.get('RMQ_PORT', 32777)

if __name__ == '__main__':

    # Create a Pipeline object
    p = Pipeline()

    # Create a Stage object
    s = Stage()

    for i in range(192):
     # Create a Task object
     t = Task()
     name = 'task'+str(i)
     t.name = name        # Assign a name to the task (optional, do not use ',' or '_')
     t.executable = ['stress --cpu 1 --timeout %ss --verbose' % random.randint(300,900)]   # Assign executable to the task

     # Add Task to the Stage
     s.add_tasks(t)

    # Add Stage to the Pipeline
    p.add_stages(s)

    # Create Application Manager
    appman = AppManager(hostname=hostname, port=port)

    # Create a dictionary describe four mandatory keys:
    # resource, walltime, and cpus
    # resource is 'local.localhost' to execute locally
    res_dict = {
        'project':'unc100',
        'resource': 'xsede.comet',
        'walltime': 15,
        'cpus': 192,
        'access_schema':'gsissh',
        'queue':'debug',
    }

    # Assign resource request description to the Application Manager
    appman.resource_desc = res_dict

    # Assign the workflow as a set or list of Pipelines to the Application Manager
    # Note: The list order is not guaranteed to be preserved
    appman.workflow = set([p])

    # Run the Application Manager
    appman.run()

Here is my radical-stack

  python               : 2.7.12
  pythonpath           :
  virtualenv           : /home/sun95/env

  radical.analytics    : 0.50.1
  radical.entk         : 0.7.10
  radical.pilot        : 0.50.19-v0.50.19-4-g70c584a@devel
  radical.utils        : 0.50.2
  saga                 : 0.50.0-v0.50.0-5-g7073412@devel

Here is the session's file. session.tar.gz

Any idea? Thank you

vivek-bala commented 5 years ago

Hey @FreddieSun , I think the maximum nodes you can get on the debug queue is 2 nodes (i.e. 48 cpu cores). See https://www.sdsc.edu/support/user_guides/comet.html#running.

You would have to either use a different queue or reduce the number of cpus you specify in the resource description.

FreddieSun commented 5 years ago

Sorry for replying late. Thanks, Vivek.