kestra-io / plugin-compress

https://kestra.io/plugins/plugin-compress/
Apache License 2.0
1 stars 2 forks source link

ArchiveCompress task unable to read from Kestra's internal storage #61

Open shrutimantri opened 5 months ago

shrutimantri commented 5 months ago

Expected Behavior

ArchiveCompress task should be able to archive compress a file from Kestra's internal storage.

Actual Behaviour

On providing the file from Kestra's internal storage, the job errors out with the following error:

2024-02-17 17:00:57.285 Unrecognized token 'kestra': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
 at [Source: (String)"kestra:///company/team/compression-1/executions/1lwBXDGJKAY9BVoBZD1wOh/tasks/httpdownload/43tZi7k6UHxj5Z9oxUz49L/17747190915210794141.csv"; line: 1, column: 7]
2024-02-17 17:00:57.285 com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'kestra': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
 at [Source: (String)"kestra:///company/team/compression-1/executions/1lwBXDGJKAY9BVoBZD1wOh/tasks/httpdownload/43tZi7k6UHxj5Z9oxUz49L/17747190915210794141.csv"; line: 1, column: 7]
    at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:2477)
    at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:760)
    at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._reportInvalidToken(ReaderBasedJsonParser.java:3041)
    at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._handleOddValue(ReaderBasedJsonParser.java:2082)
    at com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:808)
    at com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:4912)
    at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4818)
    at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3772)
    at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3755)
    at io.kestra.plugin.compress.ArchiveCompress.writeArchive(ArchiveCompress.java:91)
    at io.kestra.plugin.compress.ArchiveCompress.run(ArchiveCompress.java:74)
    at io.kestra.plugin.compress.ArchiveCompress.run(ArchiveCompress.java:27)
    at io.kestra.core.runners.Worker$WorkerThread.run(Worker.java:729)

Steps To Reproduce

  1. Use the following flow:
id: compression-1
namespace: company.team
tasks:
  - id: httpdownload
    type: io.kestra.plugin.fs.http.Download
    uri: "https://raw.githubusercontent.com/kestra-io/datasets/main/csv/products.csv"
  - id: archive_compress
    type: "io.kestra.plugin.compress.ArchiveCompress"
    from: "{{ outputs.httpdownload.uri }}"
    algorithm: ZIP
    compression: GZIP
  1. Execute the flow.

Environment Information

Example flow

id: compression-1
namespace: company.team
tasks:
  - id: httpdownload
    type: io.kestra.plugin.fs.http.Download
    uri: "https://raw.githubusercontent.com/kestra-io/datasets/main/csv/products.csv"
  - id: archive_compress
    type: "io.kestra.plugin.compress.ArchiveCompress"
    from: "{{ outputs.httpdownload.uri }}"
    algorithm: ZIP
    compression: GZIP
loicmathieu commented 1 month ago

@shrutimantri there is a FileCompress task for that, ArchiveCompress is to compress multiple files and give each file a name. I think it's a documentation issue.

anna-geller commented 2 weeks ago

@shrutimantri assigned to you to add documentation, ping me or Will if you have any questions 👍

smunteankestra commented 2 weeks ago

reproducing

image