airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
16.16k stars 4.13k forks source link

Source MySQL: different schemas with same prefix and same tables #18042

Open carlonuccio opened 2 years ago

carlonuccio commented 2 years ago

Tell us about the problem you're trying to solve

We've more than 1k schemas with same prefix and same tables on a MySQL instance. So now, we need to declare each source airbyte connection for each schema. Example:

| prefix_schema1 | --- table1 | --- table2 | prefix_schema2 | --- table1 | --- table2

Describe the solution you’d like

It could be useful to have a unique source definition to query all schemas.

Describe the alternative you’ve considered or used

I'm considering to use rest api to create dynamic source connections

Additional context

-

Are you willing to submit a PR?

Yes

YowanR commented 2 years ago

Awesome @carlonuccio! We're excited to review your PR as soon as it's ready :)