search

TonicAI / condenser

Condenser is a database subsetting tool
https://www.tonic.ai
MIT License
312 stars 48 forks source link

ignore some columns on copy_rows #28

Open csonuryilmaz opened 2 years ago

csonuryilmaz commented 2 years ago

Hi, is there a way to disable some table columns from config.json on copy_rows step?

MySQL has generated columns feature. See docs here.

So we can't insert or update value for generated columns since values of a generated column are computed from an expression included in the column definition.

We need to ignore those columns on copy_rows step. They will be filled by database automatically.

:thinking: Tool may detect those columns automatically and ignore them from select/insert process, if MySQL has some metadata to identify generated columns at information_schema.columns.

:bulb: But manually entering those kinds of table columns from config.json should be enough for most cases. Also it wil give more flexibility for some future problematic columns.

Note: I've read about a possible workaround at issue #17 and evaluate this comment for my case. But it's not a solution for generated columns.

These columns also don't accept null values. An example error message from MySQL:

... The value specified for generated column 'anniversary_date' in table 'customer' is not allowed.

Travmatth commented 2 years ago

Hi Onur, sorry for the delayed response. Unfortunately in looking through our codebase I don't see a way to operate over generated columns currently, for the reasons you raise. As you allude to, we'd need to exclude such columns entirely inside copy_rows. While I'd welcome a PR to address this, sadly it's not something we can prioritize currently.