UploadFiles task doesn't upload Namespace Files as expected

[anna-geller]

Describe the issue

I'm trying to download scripts from Git and upload them to my namespace. Both approaches resulted in a successful execution but without files being actually uploaded:

id: upload_from_git
namespace: dev

tasks:
  - id: wdir
    type: io.kestra.plugin.core.flow.WorkingDirectory
    tasks:
      - id: git
        type: io.kestra.plugin.git.Clone
        url: https://github.com/kestra-io/scripts
        branch: main

      - id: upload
        type: io.kestra.plugin.core.namespace.UploadFiles
        namespace: dev
        destination: /etl
        files: 
          - etl/

zip approach:

id: upload
namespace: dev

tasks:
  - id: download
    type: io.kestra.plugin.core.http.Download
    uri: https://github.com/kestra-io/scripts/archive/refs/heads/main.zip

  - id: unzip
    type: io.kestra.plugin.compress.ArchiveDecompress
    from: "{{ outputs.download.uri }}"
    algorithm: ZIP

  - id: upload
    type: io.kestra.plugin.core.namespace.UploadFiles
    namespace: dev
    # destination: /scripts/
    files: "{{ outputs.unzip.files }}"

Environment

• Kestra Version:
• Operating System (OS/Docker/Kubernetes):
• Java Version (if you don't run kestra in Docker):

[anna-geller]

Reopened as the examples from the issue body still don't work on develop.

Try:

id: upload2
namespace: company.team

tasks:
  - id: download
    type: io.kestra.plugin.core.http.Download
    uri: https://github.com/kestra-io/scripts/archive/refs/heads/main.zip

  - id: unzip
    type: io.kestra.plugin.compress.ArchiveDecompress
    from: "{{ outputs.download.uri }}"
    algorithm: ZIP

  - id: upload
    type: io.kestra.plugin.core.namespace.UploadFiles
    namespace: company.team
    # destination: /scripts/
    files: "{{ outputs.unzip.files }}"

[anna-geller]

This example works:

id: dbt_setup
namespace: dwh

tasks:
  - id: wdir
    type: io.kestra.plugin.core.flow.WorkingDirectory
    tasks:
      - id: git_clone
        type: io.kestra.plugin.git.Clone
        url: https://github.com/kestra-io/dbt-example
        branch: master

      - id: upload
        type: io.kestra.plugin.core.namespace.UploadFiles
        files: 
          - "glob:**/dbt/**"
        namespace: dwh

we'll open a separate issue to make the syntax more friendly

[anna-geller]

Next steps discussed in a call -- to support both files map and a RegEx, we'll make the following adjustments:

1. Add a filesMap property that can accept a map or output from downloads tasks like "{{ outputs.unzip.files }}"
2. Adjust the files property to only support Regex - for now the regex follows the Ant Style pattern so we should document that fact in the task docs + provide an example using it e.g. this dbt example - @Skraye let's add the dbt example above as a full example to UploadFiles