chop-dbhi / dataexpress

[NOT MAINTAINED] DataExpress is a simple, Scala-based cross database ETL toolkit supporting Postgres, MySql, Oracle, SQLServer, and Sqlite
http://dataexpress.research.chop.edu
BSD 2-Clause "Simplified" License
72 stars 9 forks source link

JDBC Type inference should not fail #18

Open mitalia opened 11 years ago

mitalia commented 11 years ago

When getting JDBC types from the database (e.g. for creating a table based off a query), DataExpress needs to use the metadata to ascertain what the type is. In most cases this is not an issue, but there are rare cases where this can fail. In one example case, SQLite will report a JDBC "Null" type for the column data type. For other databases, stored procedures may make it impossible to determine the type. For these scenarios, rather than total failure, DataExpress should warn that the type is not fully supported. The column on the new table should be created as a VARCHAR and then the data values should be coerced. The question remains what the view size of the VARCHAR should be, bu this likely can come from the database metadata's maximum VARCHAR size.