tenzir / public-roadmap

The public roadmap of Tenzir
https://docs.tenzir.com/roadmap
4 stars 0 forks source link

ADBC Operator #73

Open mavam opened 11 months ago

mavam commented 11 months ago

ADBC is a client-side abstraction that enables Arrow-native interacting with remote DBs. If Tenzir provides an adbc source operator, we can pull data natively into a pipeline from a variety of systems, including PostreSQL, SQLite, and Snowflake.

Since ADBC also has bulk ingestion API, it could also be a sink operator.

### Definition of Done
- [ ] Understand ADBC and its full extent of capabilities
- [ ] Design an `adbc` operator as source (and potentially sink)
- [ ] Implement and test with SQLite
- [ ] Expand to Snowflake and PostgreSQL