mintproject / MINT-Transformation

MIT License
3 stars 1 forks source link

update pipeline to detect deadlock #88

Closed raunaqtri1 closed 4 years ago

raunaqtri1 commented 4 years ago

The new streaming pipeline system has the possibility of running into a deadlock if not used correctly. With this update, we detect it and stop the pipeline before it gets stuck indefinitely.

Added a shorthand for pipeline inputs in the config schema. In addition to the standard way of defining pipeline inputs as a dictionary {"comment": "Some comment", "value": "Any valid data type"}, you can also simply specify the "value" directly if there is no comment. This is applicable only for non-dictionary values. If the "value" is a dictionary then it can only be used with the standard way. You can see the updated yml files in examples/aggregation for example