kestra-io / plugin-scripts

https://kestra.io/plugins/
Apache License 2.0
9 stars 9 forks source link

GString objects are not properly serialized #86

Closed yvrng closed 7 months ago

yvrng commented 11 months ago

Expected Behavior

I use a lot of Groovy scripts to transform datasets.

To give a little context: in Groovy, when we use string substitution, like "${key} ${value}", it creates an instance of GString (it's not String). Unfortunately, these instances are not properly serialized when written in ION files.

For example, if I have this script:

def value = "awesome"
row = "Kestra is ${value}!"

When serialized in ION, I would like to have this result:

"Kestra is awesome"

Actual Behaviour

But, I find:

{values:["awesome"],strings:["Kestra is ","!"],empty:false,valueCount:1,bytes:{{S2VzdHJhIGlzIGF3ZXNvbWUh}}}

The GString object is serialized as a POJO, instead of calling its representation with toString()

For other needs, I created this Jackson module, registrable mapper.registerModule(new GroovyModule()) :

public class GroovyModule extends SimpleModule {

    public GroovyModule() {
        addSerializer(GString.class, new GStringSerializer());
    }

    private static class GStringSerializer extends ToStringSerializerBase {

        public GStringSerializer() {
            super(GString.class);
        }

        @Override
        public String valueToString(Object value) {
            return value.toString();
        }
    }
}

Steps To Reproduce

No response

Environment Information

Example flow

id: bad-serdes
namespace: dev.yvrng
tasks:
  - id: transform
    type: io.kestra.plugin.scripts.groovy.FileTransform
    from: >
      [{"value":"awesome"}]
    script: >
      row = "Kestra is ${row.value}!"
loicmathieu commented 8 months ago

I'm afraid this is not as simple as it seems as our ObjectMapper is shared by all plugins and we don't want to add the Groovy library to all of them so we can't add a Jackson module that depends on Groovy.

By the way, you wan easily make it works by calling toString() by yourself:

id: bad-serdes
namespace: myteam
tasks:
  - id: transform
    type: io.kestra.plugin.scripts.groovy.FileTransform
    from: >
      [{"value":"awesome"}]
    script: >
      row["value"] = "Kestra is ${row.value}!".toString()
yvrng commented 7 months ago

Thanks Loïc, I understand you don't want to include Groovy globally.

We applied your solution, it works fine.