search

catalyst / moodle-tool_dataflows

A generic workflow and processing engine which can be configured to do a large variety of tasks.
GNU General Public License v3.0
10 stars 5 forks source link

In the step chooser break connectors with side effects vs not into two groups #377

Open brendanheywood opened 1 year ago

brendanheywood commented 1 year ago

This also means we have 6 groups so it will stack nicer at different screen sizes

brendanheywood commented 1 year ago

We also need to come up with a nice simple name for connectors that have actions side effects, and unless noone has a better name I think we should go with 'Actions'

keevan commented 1 year ago

Can you please elaborate on what you mean by connectors that have actions? Do you mean effects or is actions something else you have in mind?

brendanheywood commented 1 year ago

I am proposing:

connector step + side effect === 'actions'

connector step - side effect === 'connectors'

keevan commented 1 year ago

I feel like actions might be too generic. Other options I've considered:

There's a big overlap with flow steps as well, in that both flow steps and connectors can have side effects, so it would be good if the convention chosen here works also for flow steps, though they don't need to necessarily.

Then there's the bucket of steps that could have a side effect and it ultimately boils down to configuration.

brendanheywood commented 1 year ago

I'd also reconsider renaming readers and writers as they won't accurately describe some things that will end up in there, and I'm not sure they warrant the creation of yet another group. Things like the ws flow step, curl flow step and others.

So how about we have some slightly more verbose but hopefully more obvious names:

keevan commented 1 year ago

Interesting.

"Step" is a bit redundant since they are all steps technically (trigger step, connector step, etc), so perhaps:

I'm also thinking of possibly overlapping connector actions and flow actions, e.g. we have an s3 copy connector, but we could have an s3 copy flow as well (e.g. reads DB record for file name and does the copy), vs a one off copy that happens in the connector step.

The only difference (should they both exist) I see between the 2 is how they are configured. Logically they function more or less the same, e.g. run the step based on some configuration which could come from an expression. Connectors return true/false, flows pass the $input through, abort or stop the flow. Why shouldn't you be able to run a web service call as a once off connector step, potentially using data received from a webhook / event trigger?

It's late and I might have forgotten but how about this further suggestion:

Wait.. curl get? Why can't a flow call a curl get :')

brendanheywood commented 1 year ago

+10

I've been grappling with that same question since the beginning around whether we really need to duplicate each of the actions or if they can be one and the same. I really like the idea of a suite of actions that could be a connector action or a flow action and you just tick a common tick box which converts all of them and ideally each one doesn't need to do any extra code for it to just work.

I'm trying to think of flow actions where it needs some extra config or special code to do things differently to a connector action but I haven't come up with anything yet.

brendanheywood commented 1 year ago

Going to defer this until after https://github.com/catalyst/moodle-tool_dataflows/issues/422 is resolved. Once we no longer care about whether something is a connector or flow action then, and we accumulate a large number of actions, it will probably make more sense to group them by function or plugin or whatever.