Open ovidals opened 10 months ago
Thanks for reporting @ovidals!
What is the result you'd expect from mapping a stream with an empty schema?
Thanks for reporting @ovidals!
What is the result you'd expect from mapping a stream with an empty schema?
Hi @edgarrmondragon , I guess if a pipeline step does not receive anything as an input, should output also an empty stream right?
For example, I had a pipeline with an extractor and a transformer before adding the mapper, and if the extractor does not extract anything, an empty stream is sent to transformer, so transformer does nothing and the pipeline ends successfully without any processing. Adding an intermediate step in the pipeline like a mapper in this case, I would expect the pipeline to behave the same, but without crashing.
Does that make sense?
Thank you!
Fixing this should be as simple as using .get(...)
in https://github.com/meltano/sdk/blob/46c14f6003baf658be0dcc25f1307c7007f5be27/singer_sdk/mapper.py#L489-L490
Hi, I'm facing an error when the mapper does not receive any content from the stream:
This is my config:
This is getting content from a CSV file extracted with "tap-sftp" plugin.
When there is no CSV file in folder, stream is empty and I get this error:
Is there any way to configure mapper to avoid crashing with empty content?
Thank you.