hyperflow-wms / hyperflow

HyperFlow: a scientific workflow engine
MIT License
65 stars 31 forks source link

RemoteJobConnector - avoidance of duplicated redis clients #42

Closed koxu1996 closed 4 years ago

koxu1996 commented 4 years ago

This PR changes they way of communication with remote jobs, that HyperFlow could use only one redis connection. Instead of using n blocking redis clients, we use single non-blocking command that is executed in loop - every checkInterval - and it resolves all relevant waiting getJobResult promises.

With such changes getJobResult behaviour is identical, so there is no need to update functions or examples. However remote job executor must be adjusted accordingly - see https://github.com/hyperflow-wms/hyperflow-job-executor/pull/6.