Add SQL Server frontend support - Githubissues

gluent / goe

GOE: a simple and flexible way to copy data from an Oracle Database to Google BigQuery.

Apache License 2.0

8 stars 2 forks source link

Add SQL Server frontend support #138

Open nj1973 opened 3 months ago

nj1973 commented 3 months ago

There is a skeleton already in place for this but be prepared to revisit all steps.

High level tasks:

Update connectivity, we should probably switch from the currently library to pyodbc
Review frontend to canonical column mappings
Implement Spark based reader, spend time looking at efficient read partitioning
Look at HWM management for partitioned table support
Predicate based offload
Final data validation