future work: delete "active flow" collection after flow ends

[robludwig]

As explained in the DB section of the manual, we have a new collection when users run a workflow, which we overwrite every time users run that workflow. We should consider

• summarizing the status of the collection when the flow finishes and then
• deleting the collection to save space

[robludwig]

In fact, we could probably harmonize the database a little more. We really only need one workflow collection for definitions and one for active, with a discriminator key called "workflow" that says which node each one is related to. This would need some work if we wanted to be able to run multiple flows at once though. Creating the active collection uses mongodb's $out feature which creates a collection and always overwrites it

[laszewsk]

Remember at one point we will run multiple workflows at the same time

On May 2, 2019, at 3:29 PM, robludwig notifications@github.com wrote:

In fact, we could probably harmonize the database a little more. We really only need one workflow collection for definitions and one for active, with a discriminator key called "workflow" that says which node each one is related to. This would need some work if we wanted to be able to run multiple flows at once though. Creating the active collection uses mongodb's $out feature which creates a collection and always overwrites it

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/cloudmesh/cloudmesh-flow/issues/25#issuecomment-488801931, or mute the thread https://github.com/notifications/unsubscribe-auth/AADHYVIQGOXXEB4QGYHTK4LPTM6K7ANCNFSM4HKFBPYQ.

[robludwig]

yes, the current setup supports that but is inefficient. Future work could make it more efficient as described above, but would have to preserve this functionality

perhaps one collection for all flows, plus one collection per running flow