jvalue / jayvee

Jayvee is a domain-specific language and runtime for automated processing of data pipelines
https://jvalue.github.io/jayvee/
117 stars 11 forks source link

[BUG] no execution result in composite blocktype #591

Open TungstnBallon opened 3 months ago

TungstnBallon commented 3 months ago

Steps to reproduce

run this model:

pipeline pip {

    XTractor
        -> TextFile
        -> CSV
        -> Table
        -> ToDecimals
        -> Loader;

    block XTractor oftype LocalFileExtractor {
        filePath: "data.csv";
    }

    block TextFile oftype TextFileInterpreter { }
    block CSV oftype CSVInterpreter { }

    block Table oftype TableInterpreter {
        header: false;
        columns: [
            "c" oftype text,
        ];
    }

    block ToDecimals oftype DecimalParser2 {
        columnName: "c";
    }

    block Loader oftype SQLiteLoader {
        table: "table";
        file: "out.sqlite";
    }
}

composite blocktype DecimalParser2 {
    input WithTextColumn oftype Table;
    output WithDecimalColumn oftype Table;

    property columnName oftype text;

    WithTextColumn
        -> ParseDecimalBlock
        -> WithDecimalColumn;

    transform ParseDecimal {
        from decText oftype text;
        to dec oftype decimal;

        dec: asDecimal decText;
    }

    block ParseDecimalBlock oftype TableTransformer {
        inputColumns: [
            columnName
        ];
        outputColumn: columnName;

        uses: ParseDecimal;
    }
}

data.csv:

"32.0"

Description

This error does not occur if the composite blocktype is replaced with a regular TableTransformer inside the pipeline