Open mscheremetjew opened 4 years ago
@mscheremetjew , I think this because the concurrent database connections exceed the Hibernate connection pool size (default 20), as a temporary way, you may increase the pool size at https://github.com/IBMSpectrumComputing/cwlexec/blob/master/src/main/java/com/ibm/spectrumcomputing/cwl/exec/util/DatabaseManager.java#L84, e.g.
properties.put("hibernate.connection.pool_size", "50");
We may need to limit (e.g. use Semaphore) the database concurrent request in future at https://github.com/IBMSpectrumComputing/cwlexec/blob/master/src/main/java/com/ibm/spectrumcomputing/cwl/exec/service/CWLInstanceService.java#L80
Does it make sense to make this a setting in the users flow somewhere? Is that supported in the Specification?
Does it make sense to make this a setting in the users flow somewhere? Is that supported in the Specification?
There is not a specific configuration option for this in the CWL standards. CWLEXEC
could detect the number of steps
in class: Workflow
and configure itself accordingly. Ideally site specific configuration is not in a CWL description. However, if it is needed, then a cwlexec:hibernate.connection_pool_size
CWL directive could be implemented by CWLEXEC
We do bump into this issue whenever we try to execute workflows in CWLEXEC with more than 20 steps, so 21 steps for instance. Also sometimes CWLEXEC hangs after it has reported the error. Could it be that sessions are not closed properly after each database transaction? We created a simple test workflow so it becomes easy to reproduce for you guys.
Here is the workflow: test-workflow.zip
This is the command we are running:
CWLEXEC reports the following and exits or sometimes just hangs: