Some codependent inputs are displayed when they shouldn't, and some SELECT values are not not displayed yet

[anna-geller]

Describe the issue

Demo of the issue: https://share.descript.com/view/zoM6oaqnBB2

To reproduce create KV:

id: add_kv_pairs
namespace: company.myteam

tasks:
  - id: access_permissions
    type: io.kestra.plugin.core.kv.Set
    key: "{{ task.id }}"
    kvType: JSON
    value: |
      ["Admin", "Developer", "Editor", "Launcher", "Viewer"]

  - id: saas_applications
    type: io.kestra.plugin.core.kv.Set
    key: "{{ task.id }}"
    kvType: JSON
    value: |
      ["Slack", "Notion", "HubSpot", "GitHub", "Jira"]

  - id: development_tools
    type: io.kestra.plugin.core.kv.Set
    key: "{{ task.id }}"
    kvType: JSON
    value: |
      ["Cursor", "IntelliJ IDEA", "PyCharm Professional", "Datagrip"]

  - id: cloud_vms
    type: io.kestra.plugin.core.kv.Set
    key: "{{ task.id }}"
    kvType: JSON
    value: |
      {
        "AWS": ["t2.micro", "t2.small", "t2.medium", "t2.large"],
        "GCP": ["f1-micro", "g1-small", "n1-standard-1", "n1-standard-2"],
        "Azure": ["Standard_B1s", "Standard_B1ms", "Standard_B2s", "Standard_B2ms"]
      }

Then, run the large flow:

id: request_resources
namespace: company.myteam

inputs:    
  - id: resource_type
    displayName: Resource Type
    type: SELECT
    required: true
    values:
      - Access permissions
      - SaaS application
      - Development tool
      - Cloud VM

  - id: access_permissions
    displayName: Access Permissions
    type: SELECT
    expression: "{{ kv('access_permissions') }}"
    allowInput: true
    dependsOn: 
      inputs: 
        - resource_type
      condition: "{{ inputs.resource_type equals 'Access permissions' }}"

  - id: saas_applications
    displayName: SaaS Application
    type: MULTISELECT
    expression: "{{ kv('saas_applications') }}"
    allowInput: true
    dependsOn: 
      inputs: 
        - resource_type
      condition: "{{ inputs.resource_type equals 'SaaS application' }}"

  - id: development_tools
    displayName: Development Tool
    type: SELECT
    expression: "{{ kv('development_tools') }}"
    allowInput: true
    dependsOn: 
      inputs: 
        - resource_type
      condition: "{{ inputs.resource_type equals 'Development tool' }}"

  - id: cloud_provider
    displayName: Cloud Provider
    type: SELECT
    values:
      - AWS
      - GCP
      - Azure
    dependsOn: 
      inputs: 
        - resource_type
      condition: "{{ inputs.resource_type equals 'Cloud VM' }}"

  - id: cloud_vms
    displayName: Cloud VM
    type: SELECT
    expression: "{{ kv('cloud_vms')[inputs.cloud_provider] }}"
    allowInput: true
    dependsOn: 
      inputs: 
        - resource_type
        - cloud_provider
      condition: "{{ inputs.resource_type equals 'Cloud VM' }}"

  - id: region
    displayName: Cloud Region
    type: SELECT
    description: The expression below provides alternative to storing the regions in KV store JSON with cloud provider as key and a list of regions as value
    expression: |
      {% if inputs.cloud_provider == 'AWS' %}
        ["us-east-1", "us-west-1", "us-west-2", "eu-west-1"]
      {% elif inputs.cloud_provider == 'GCP' %}
        ["us-central1", "us-east1", "us-west1", "europe-west1"]
      {% else %}
        ["eastus", "westus", "centralus", "northcentralus"]
      {% endif %}
    dependsOn: 
      inputs: 
        - resource_type
        - cloud_provider
        - cloud_vms
      condition: "{{ inputs.resource_type equals 'Cloud VM' }}"

tasks:
  - id: send_approval_request
    type: io.kestra.plugin.notifications.slack.SlackIncomingWebhook
    url: https://reqres.in/api/slack
    payload: |
      {
        "channel": "#devops",
        "text": "Validate resource request: `{{ inputs }}`. To approve the request, click on the `Resume` button here http://localhost:28080/ui/executions/{{flow.namespace}}/{{flow.id}}/{{execution.id}}"
      }

  - id: wait_for_approval
    type: io.kestra.plugin.core.flow.Pause
    onResume:
      - id: approved
        description: Whether to approve the request
        type: BOOLEAN
        defaults: true

      - id: comment
        description: Extra comments about the provisioned resources
        type: STRING
        defaults: All requested resources are approved

  - id: approve
    type: io.kestra.plugin.core.http.Request
    uri: https://reqres.in/api/resources
    method: POST
    contentType: application/json
    body: "{{ inputs }}"

  - id: log
    type: io.kestra.plugin.core.log.Log
    message: Status of the request {{ outputs.wait_for_approval.onResume.comment }}. Process finished with {{ outputs.approve.body }}

Environment

• Kestra Version:
• Operating System (OS/Docker/Kubernetes):
• Java Version (if you don't run kestra in Docker):

[anna-geller]

it might be a separate issue, but I noticed that even when selecting SaaS applications in the first option and explicitly marking all other inputs as required: false, Kestra still tries to render other inputs e.g. select Slack + GitHub and hit Execute and you will see the error:

At cloud_vms: Cannot render 'expression'. Cause: Unable to find inputs used in the expression {{ kv('cloud_vms')[inputs.cloud_provider] }} at line 1

[fhussonnois]

@anna-geller this should be mostly resolved, but there is a new issue with :

  - id: cloud_vms
    type: io.kestra.plugin.core.kv.Set
    key: "{{ task.id }}"
    kvType: JSON
    value: |
      {
        "AWS": ["t2.micro", "t2.small", "t2.medium", "t2.large"],
        "GCP": ["f1-micro", "g1-small", "n1-standard-1", "n1-standard-2"],
        "Azure": ["Standard_B1s", "Standard_B1ms", "Standard_B2s", "Standard_B2ms"]
      }

In the KV the entry is stored as a STRING not a JSON.

[anna-geller]

nope, I can't reproduce on develop, seems to work perfectly fine as JSON, can you repull and try again?

id: myflow
namespace: company.team
tasks:
  - id: cloud_vms
    type: io.kestra.plugin.core.kv.Set
    key: "{{ task.id }}"
    kvType: JSON
    value: |
      {
        "AWS": ["t2.micro", "t2.small", "t2.medium", "t2.large"],
        "GCP": ["f1-micro", "g1-small", "n1-standard-1", "n1-standard-2"],
        "Azure": ["Standard_B1s", "Standard_B1ms", "Standard_B2s", "Standard_B2ms"]
      }

[fhussonnois]

You're right, it works better after rebasing!

Also, there is an issue with the input region in your example.

Bellow is the pebble expression that should work, but it's not as trivial as expected.

  - id: region
    displayName: Cloud Region
    type: SELECT
    description: The expression below provides alternative to storing the regions in KV store JSON with cloud provider as key and a list of regions as value
    expression: |-
      {% if inputs.cloud_provider == 'AWS' %}
      {{- ["us-east-1", "us-west-1", "us-west-2", "eu-west-1"] -}}
      {% elseif inputs.cloud_provider == 'GCP' %}
      {{- ["us-central1", "us-east1", "us-west1", "europe-west1"] -}}
      {% else %}
      {{- ["eastus", "westus", "centralus", "northcentralus"] -}}
      {% endif %}

However, we have an internal error thrown by our pebble renderer that prevent it to be rendered properly as an array :Tried to add ArrayList<String> to java.lang.String .

As far as I understand, pebble want to output a first empty character somewhere for that following expression.

A hack seems possible to get around this error, but I don't know what the consequences might be for other plugins.

For the moment, I would recommand to not make any modification on the pebble rendering for solving that. For such use-case, an alternative, could be to define one region input for each provider. We should use the cooldown period to enhance our pebble support with expression returning strong typed object.

[anna-geller]

all good, we can recommend sticking to KV pairs for this, that pebble example is not the prettiest anyway 👍

[anna-geller]

this example is now fully working on develop, closing the issue