kestra-io / plugin-serdes

https://kestra.io/plugins/plugin-serdes/
Apache License 2.0
2 stars 6 forks source link

XmlReader always returns an empty file #69

Closed yvrng closed 10 months ago

yvrng commented 11 months ago

Expected Behavior

When I use the XmlReader I want my XML file to be parsed as expected

Actual Behaviour

The result ION file is empty, whatever the content of query

Steps To Reproduce

No response

Environment Information

Example flow

id: empty-xml
namespace: dev.yvrng
inputs:
  - name: from
    type: FILE
tasks:
  - id: read
    type: io.kestra.plugin.serdes.xml.XmlReader
    from: "{{ inputs.from }}"
    query: /root/pokemon
    parserConfiguration:
      forceList:
        - "type"

with the following dataset:

<root>
    <pokemon id="001" name="Bulbasaur">
        <type index="0">grass</type>
        <type index="1">poison</type>
        <height>0.7</height>
        <weight>6.9</weight>
    </pokemon>
    <pokemon id="004" name="Charmander">
        <type index="0">fire</type>
        <height>0.6</height>
        <weight>8.5</weight>
    </pokemon>
    <pokemon id="007" name="Squirtle">
        <type index="0">water</type>
        <height>0.5</height>
        <weight>9.0</weight>
    </pokemon>
</root>
yvrng commented 11 months ago

This regression may have been introduced with version 0.13.4, there were no issues before.

But, I'm not able to reproduce it with unit tests:

@Test
void readPokemon() throws Exception {
    // given
    var sourceFile = SerdesUtils.resourceToFile("xml/pokemon.xml");
    var sourceUri = this.serdesUtils.resourceToStorageObject(sourceFile);

    // when
    var task = XmlReader.builder()
            .id("read")
            .type(XmlReader.class.getName())
            .from("{{ inputs.from }}")
            .query("/root/pokemon")
            .parserConfiguration(XmlReader.ParserConfiguration.builder()
                    .forceList(Set.of("type"))
                    .build())
            .build();

    Map<String, Object> inputs = Map.of("from", sourceUri);
    var output = task.run(TestsUtils.mockRunContext(this.runContextFactory, task, inputs));

    // then
    final List<String> targetContentList;
    try (InputStream in = runContextFactory.of().uriToInputStream(output.getUri()); BufferedReader reader = new BufferedReader(new InputStreamReader(in))) {
        targetContentList = reader.lines().toList();
    }
    assertThat(targetContentList, hasSize(3));
}
Skraye commented 11 months ago

Hi ! I can't reproduce your issue : Using your file & your flow : image

As you're using K8S*, are you using the kestra image -full version that include all plugins, or are you adding plugins by yourself ?

yvrng commented 11 months ago

Hi! Yes, we are using the full image as basis, then we add our custom plugin within it.

yvrng commented 11 months ago

We're no longer able to reproduce the bug on our side, either. Everything seems to be OK now, but no idea what might have caused the problem.

yvrng commented 10 months ago

My team notified me that we still have the issue in dev (v0.13.5)