kestra-io / plugin-fs

https://kestra.io/plugins/plugin-fs/
Apache License 2.0
6 stars 7 forks source link

Allow spaces in file names when using io.kestra.plugin.fs.sftp.trigger #147

Closed KoenWindey closed 1 month ago

KoenWindey commented 2 months ago

Expected Behavior

When a file name has a space (eg. "my file name.txt"), I would like the SFTP (and by extension all of File System plugin features) to not throw an error, but handle the spaces correctly.

Actual Behaviour

Instead, I get this error:

java.lang.IllegalArgumentException: Illegal character in path at index 46: /home/stark/docker/dropbox/data/archive/sstark logo long.webp at java.base/java.net.URI.create(Unknown Source) at java.base/java.net.URI.resolve(Unknown Source) at io.kestra.plugin.fs.vfs.VfsService.move(VfsService.java:213) at io.kestra.plugin.fs.vfs.VfsService.performAction(VfsService.java:270) at io.kestra.plugin.fs.vfs.Trigger.evaluate(Trigger.java:154) at io.kestra.core.runners.WorkerTriggerThread.doRun(WorkerTriggerThread.java:24) at io.kestra.core.runners.AbstractWorkerThread.run(AbstractWorkerThread.java:57) Caused by: java.net.URISyntaxException: Illegal character in path at index 46: /home/stark/docker/dropbox/data/archive/sstark logo long.webp at java.base/java.net.URI$Parser.fail(Unknown Source) at java.base/java.net.URI$Parser.checkChars(Unknown Source) at java.base/java.net.URI$Parser.parseHierarchical(Unknown Source) at java.base/java.net.URI$Parser.parse(Unknown Source) at java.base/java.net.URI.<init>(Unknown Source) … 7 more 2024-09-07 15:32:54.210 trigger

Steps To Reproduce

  1. Set up a trigger with SFTP monitoring on new file
  2. Use the 'MOVE' option
  3. Load a file with blank spaces in its name
  4. Watch the trigger throwing an error about 'illegal characters'

Environment Information

Example flow

id: test_filechange
namespace: company.team
tasks:
  - id: for_each_file
    type: io.kestra.plugin.core.flow.EachSequential
    value: "{{ trigger.files }}"
    tasks:
      - id: log
        type: io.kestra.plugin.core.log.Log
        level: WARN
        message: "File found: {{taskrun.value | jq('.name')}}"

triggers:
  - id: trigger
    type: io.kestra.plugin.fs.sftp.Trigger
    host: kestra.host.ip
    username: myuser
    password: mypass
    action: MOVE
    rootDir: false
    from: "/data/in/"
    moveDirectory: "/data/archive/"
    interval: PT10S
    conditions: []
    labels: []
    stopAfter: []
Ben8t commented 1 month ago

@mgabelle here are some docker snippets to emulate a FTP server locally

  docker container run -d \
    -p 21:21 \
    -p 21000-21010:21000-21010 \
    -e USERS="admin|kestra" --network kestra-net \
    --name local_ftp \
    delfer/alpine-ftp-server

if you want to log in with a shell

  docker exec -it local_ftp /bin/sh

Corresponding Kestra trigger

triggers:
  - id: trigger-ftp
    type: io.kestra.plugin.fs.ftp.Trigger
    host: host.docker.internal
    port: "21"
    username: one
    password: "1234"
    from: "./input/"
    interval: PT1S
    action: MOVE
    moveDirectory: "./output/"
nicoallgood commented 1 month ago

If needed, I've also found this Docker image to be helpful: https://hub.docker.com/r/atmoz/sftp/ 👍