Prevent column misalignment, NaN errors and other type incompatibilities also for --sqlite option, by:
1) determining a reference schema for the table columns
2) opening a writer
3) converting all subsequent files to that reference schema)
The functions are already deployed for the --parquet option (merged PR "Parquet_integration #122). This issue hence boils down to hooking existing functionality up for the 'sqlite' backend.
Prevent column misalignment, NaN errors and other type incompatibilities also for
--sqlite
option, by: 1) determining a reference schema for the table columns 2) opening a writer 3) converting all subsequent files to that reference schema)The functions are already deployed for the
--parquet
option (merged PR "Parquet_integration #122). This issue hence boils down to hooking existing functionality up for the 'sqlite' backend.