Closed TungstnBallon closed 2 months ago
Execute this model (big_file.csv is larger than 2GB):
big_file.csv
pipeline Pipeline { Extractor -> ToTextFile -> ToCSV -> ToTable -> Loader; block Extractor oftype LocalFileExtractor { filePath: "./big_file.csv"; } block ToTextFile oftype TextFileInterpreter { } block ToCSV oftype CSVInterpreter { enclosing: '"'; } block ToTable oftype TableInterpreter { header: true; columns: [ "C1" oftype integer, "C2" oftype text, "C3" oftype boolean, ]; } block Loader oftype SQLiteLoader { table: "Data"; file: "./Data.sqlite"; } }
Actual: The interpreter fails with a "file not found" message:
[Pipeline] Overview: Blocks (5 blocks with 1 pipes): -> Extractor (LocalFileExtractor) -> ToTextFile (TextFileInterpreter) -> ToCSV (CSVInterpreter) -> ToTable (TableInterpreter) -> Loader (SQLiteLoader) error: File './big_file.csv' not found. $In /home/jonas/Code/uni/hiwi/jayvee/pipeline.jv:14:2 14 | block Extractor oftype LocalFileExtractor { | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 15 | filePath: "./big_file.csv"; | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 16 | } | ^^^^^
Steps to reproduce
Execute this model (
big_file.csv
is larger than 2GB):Description
Actual: The interpreter fails with a "file not found" message: