As an ETLHelper user, I want it explicitly pointed out that load assumes that all items have the same columns, so that I don't miss any when I load heterogeneous data.
Description
The load function peeks at the first row of data to identify the column names used to autogenerate the insert query.
If later items in the data contain extra columns, those columns will not be inserted.
The process will continue without raising any error.
I don't think there is an easy way to check for each item, but we should explicitly make this clear in the documentation.
Acceptance criteria
[ ] README.md explains that load looks at the first item to determine the column names for the insert query, so if later items have extra columns, that data will not be loaded.
Summary
As an ETLHelper user, I want it explicitly pointed out that
loadassumes that all items have the same columns, so that I don't miss any when I load heterogeneous data.Description
The
loadfunction peeks at the first row of data to identify the column names used to autogenerate the insert query. If later items in the data contain extra columns, those columns will not be inserted. The process will continue without raising any error.I don't think there is an easy way to check for each item, but we should explicitly make this clear in the documentation.
Acceptance criteria
loadlooks at the first item to determine the column names for the insert query, so if later items have extra columns, that data will not be loaded.