search

TonicAI / condenser

Condenser is a database subsetting tool
https://www.tonic.ai
MIT License
315 stars 47 forks source link

Redundancy between db_name and prefacing every table with the name of the database? #34

Open derekyle opened 2 years ago

derekyle commented 2 years ago

First, thank you for this project!

Perhaps I am missing something, but it seems redundant to me to require . in the config schema. We have to indicate the db_name in the source_db_connection_info clause so it seems that db_name should be assumed from that point on. Or is there an ability to copy from multiple source databases somehow?

derekyle commented 2 years ago

Also, wildcard db names or regex would be a killer feature. We have a lot of tables that end in _log. It would be nice to be able to indicate that all tables ending in _log could be ignored.

mwcaisse commented 1 year ago

I agree it is redundant to also have to require the database name prefix on every table for MySQL. It is a byproduct of supporting PostgreSQL, where a database can contain multiple schemas, in which case each table would need to have the schema prefix as it can be different. However this isn't true for MySQL as schema and database are synonymous and you can only connect to one at a time.

The project could definitely be refactored to not have to specify the database/schema prefix for every table on MySQL, however we don't have the bandwidth to take on new features for this project currently, but do welcome contributions. If you want to make the change and submit a PR we would be happy to review and merge it.

Supporting wild card or regexes for table names would be interesting and possibly a bit more complicated. At least MySQL lets you add special characters to your table name, for example hello_.* is a valid table name in MySQL. There would need to be a way to differentiate regex expressions from verbatim table names. Would welcome a PR to add support for it though.