search

opensearch-project / flow-framework

OpenSearch plugin that enables builders to innovate AI apps on OpenSearch
Apache License 2.0
31 stars 29 forks source link

[FEATURE] Allow provisioning of a completed workflow with no resources #537

Open dbwiddis opened 6 months ago

dbwiddis commented 6 months ago

Is your feature request related to a problem?

Currently, we prevent provisioning a workflow twice. The intent of this restriction is to avoid duplicating resources. It's possible to provision a second time if you deprovision first. However, in the edge case where a workflow does not have any resources, the "deprovision" step is a no-op and should not be required.

What solution would you like?

The FlowFrameworkIndicesHandler currently has a boolean to check whether a workflow is started: https://github.com/opensearch-project/flow-framework/blob/main/src/main/java/org/opensearch/flowframework/indices/FlowFrameworkIndicesHandler.java#L491

This is checked in two REST actions to prevent modification: updating (PUT) a workflow and provisioning a workflow.

In both cases, if the provisioning is complete and there are no resources, there is no reason to prevent editing/provisioining; essentially deprovisioning (no-op) to reset the state.

This method should be updated to allow a COMPLETED provisioning with no resources to be considered the same as "not started". For performance reasons, the not started check should be done first, as the COMPELTED check requires an additional query of the state index to determine whether resources were provisioned.

What alternatives have you considered?

We may just want to allow a parameter, e.g, ?deprovision=true which would complete the deprovisioning (if any) prior to provisioning a second time. This would work well with the future "fine grained provisioning" feature where in some cases only a partial deprovisioning is required.

To eliminate the need for a resource check for a COMPLETED provision, we can consider another workflow state that we set when provisioning is complete and we have no resources, or we can rename NOT_STARTED to some neutral term, e.g., PROVISIONABLE, meaning there are no resources.

Do you have any additional context?

With the existing ML configuration steps, we don't expect this edge case often, but with automation steps which do not provision, as envisioned by #522, this may be a common situation.

dbwiddis commented 6 months ago

Given the potential for "fine grained provisioining" in the future, it seems the "alternative you have considered" is the best option. Allow a parameter like "deprovision=true" to fully or partially deprovision.

xinlamzn commented 5 months ago

Is this a feature or bug-fix?

dbwiddis commented 5 months ago

Is this a feature or bug-fix?

More of a feature as there's not really a bug. Provisioning a workflow that doesn't create resources is somewhat of an edge case. It's worth seeing how we fully integrate versioning with the front end before committing to a specific solution.