Change the value of the execution's inputs before starting a Replay

[Ben8t]

Feature description

The use case is rather simple:

• to have a flow requiring input parameters
• an execution of this flow successfully processes n tasks
• the execution fails at a (n+1) task
• the reason for the failure was an error in the input parameter
• now you are able to Replay tasks of the execution, but you are unable to change/fix the incorrect input parameter and then Replay => start an entirely new execution

Being able to edit input parameters && Replay the tasks "from the middle of flow" would be great for flows featuring long running and/or non-idempotent tasks.

cc @yuri1969

[iamviqar]

@Ben8t I would like to attempt this

[MilosPaunovic]

It's all yours @iamviqar! :rocket:

[iamviqar]

Thanks @MilosPaunovic , do we any sample flow or reproducible steps to get started on this

[MilosPaunovic]

@Ben8t would be the best person to provide the sample flow here.

[Ben8t]

Here is a reproducer:

id: replay_inputs
namespace: company.team

inputs:
  - id: test
    type: STRING
    required: true
tasks:
  - id: log1
    type: io.kestra.plugin.core.log.Log
    message: '{{inputs.test}}'

  - id: if
    type: io.kestra.plugin.core.flow.If
    condition: '{{ inputs.test == "Hello" }}'
    then:
      - id: log2
        type: io.kestra.plugin.core.log.Log
        message: "correct input"

    else:
      - id: error
        type: io.kestra.plugin.core.execution.Fail   

Here we would like to being able to edit input parameters and replay the failing tasks

[iamviqar]

Thanks @Ben8t

[iamviqar]

@Ben8t just trying to understand the existing functionality, when failed execution is replayed is it re-initiated from the intermediate failure point or from start ?

[Ben8t]

when failed execution is replayed is it re-initiated from the intermediate failure point or from start ?

Failed execution can be replayed at the task level (where it fails generally) or globally (re-run the full execution). In both cases the users would like be able to change (or not) the flow inputs

[iamviqar]

Quick update, I've started working on this made some changes on UI to incorporate input collection on replay dialogue.

@Ben8t Please confirm if this screen is as expected

[iamviqar]

@Ben8t I see restart option on overview screen of execution is also not prompted with input. Do you think this also needs to be fixed?

tagging @anna-geller @MilosPaunovic

[Ben8t]

Thanks @iamviqar for the work in progress 👍

Yes, I think it would be a great addition too as it restart the execution from the FAILED task (as highlighted by the tooltip)

From the the first screenshot you shared, would it be possible to prefill the input value with the first one filled during the prior execution please ?

I can imagine a flow with many inputs, so as a user I won't be happy to fill everything again if I only need to rerun the execution (without changing any inputs, or only changing one out of of many)

[iamviqar]

Sure @Ben8t , let me attempt to get restart functionality along with this issue and FYI I am already attempting to pre-populate the input values

[iamviqar]

I’ve implemented changes to display the input form in the replay and restart dialogue boxes.

I updated the UI code to pass the values collected from the input form and created a new API endpoint, /restart-with-input, to avoid altering the existing /replay API. Please let me know if you think it would be better to modify the /replay API instead of creating a new one.

On the API side, I’m successfully receiving the input values from the UI as method parameters. However, I’m having difficulty figuring out how to apply these newly collected inputs to the existing execution, and it's taking me a while to understand the process. I would appreciate some assistance from a backend developer to move forward.

You can refer to this commit for my API code: https://github.com/iamviqar/kestra/commit/a24d3edef7752603a1be05d34dc1d7ae77cfa65e

tagging @loicmathieu @tchiotludo @Skraye @MilosPaunovic @anna-geller

[MilosPaunovic]

Hey @iamviqar, is there a PR opened for this, as I can not seem to find it?

[anna-geller]

same suggestion as Milos, perhaps you can share a PR and we can help suggest what's missing to get it to the finish line?

[iamviqar]

Here is the Draft PR: https://github.com/kestra-io/kestra/pull/5698

[MilosPaunovic]

@iamviqar What is the status of this PR?

[iamviqar]

@MilosPaunovic I need to make progress based on reference code provided by @Skraye on backend implementation.

[Ben8t]

@iamviqar any news on this ?