Open chuwy opened 4 years ago
We need to be careful with this, as this can hide a parsing/incompatibility problem that needs to be addressed.
If we implement this I think that we should add a boolean to the parsing function to tell if we want to use a fallback or not.
Both are very good points.
Another downside is that we'll be limited in structure flexibility for all bad rows. Also (or "on the other hand"), I think consumer should always be explicit about what bad rows it needs.
We can make
GenericError
a some kind of "base class" for all bad rows, so if producer (enrich
) produces new kind of bad rows that consumer (loader
) cannot recognize yet - instead of failing it could attempt to parse it asGenericError
.28 #47