Processes run in OPT mode are not immediately running after predecessors have finished #25

[PropPortfolio]

Hi @MariusWirtz , I am facing similar issue as @lbrowncubewise . Let's consider the example attached in "SnapshotAllCubes.txt" file above.

id="11" predecessors="" instance="XXfab" process="}bedrock.cube.data.export" id="21" predecessors="" instance="XXfab" process="}bedrock.cube.data.clear" id="31" predecessors="11,21" instance="XXfab" process="}bedrock.cube.data.import" id="12" predecessors="" instance="XXfab" process="}bedrock.cube.data.export" id="22" predecessors="" instance="XXfab" process="}bedrock.cube.data.clear" id="32" predecessors="12,22" instance="XXfab" process="}bedrock.cube.data.import" id="13" predecessors="" instance="XXfab" process="}bedrock.cube.data.export" id="23" predecessors="" instance="XXfab" process="}bedrock.cube.data.clear" id="33" predecessors="13,23" instance="XXfab" process="}bedrock.cube.data.import" etc...

My understanding is that, If I am running above in OPT mode with 3 maximum workers,

In First run it should execute, ID # 11, 21 & 12 Lets assume ID # 11 finished first, then It should trigger ID # 22 (while ID # 21 & 12 still going). Next lets assume ID # 21 finished, I would expect ID # 31 to kick off ( as ID #11 & 21 finished), But what it does is go to ID # 13 and so on to ID # 23.

ID # 31, 32 & 33 will wait until all other ID # without predecessors are finished executing, while I was expecting RushTI should have triggered them in between as predecessors IDs are completed.

Originally posted by @CognosTM1 in https://github.com/cubewise-code/rushti/issues/25#issuecomment-658786059

[PropPortfolio]

SnapshotAllCubes.txt RushTI.log

[MariusWirtz]

Hi @CognosTM1,

it works as designed. The OPT mode thinks in terms of levels. In your case the levels would be: level1: 11, 21, 12, 22, 23 level2: 31, 32, 33

I agree this is not ideal. Feel free to create a MR to improve the OPT mode.