new gbak option to ignore specific data in specific tables during restore (and backup)

[atronah]

We in our work need new gbak option, which provides ability to skip some data (some rows) of specific tables during restore database from backup file.

It helps us with making lightweight production database copy to test our software update before doing that update in production database. Out production database stores data collected over 10 years (and use almost 1Tb disk space) and 90% of that data isn't required in test enviroment.

I see this option like bellow:

gbak ... -skip_rows "MY_BIG_TABLE: created <dateadd(-6 month to current_date)" -skip_rows "MY_ANOTHER_BIG_TABLE: is_archive > 0"

[aafemt]

FBExport has such options. Why can't you use it instead of gbak?

[atronah]

FBExport has such options. Why can't you use it instead of gbak?

Because it is not a standard tool of firebird and cannot be easy installed on production servers (and it does not inspire confidence). We prefer to use standard tools for core actions like backup and restore, to prevent unpleasant surprises.

[aafemt]

Why not create your own utility for your own task?

[atronah]

Because I don't have enough skill to re-implement full gbak functionality with adding some small feature requested above. And the second reason I've mentioned above: we prefer to use official tools from DBMS developers to reduce risks of db damage because of unaccounted things.

Why was the option -SKIP_DATA added without suggesting to "create own utility"?

I think, requested feature could be useful not only for me.

[aafemt]

I implemented it because it was fun and simple. Your request requires reworking whole way gbak retrieve data to add custom filtration which is not simple and open whole bunch of problems with SQL injection which is not fun.

BTW, specialized data pump utility for particular database doesn't need a lot of skill. It is versatility that makes troubles.

[romansimakov]

Feature requester doesn't have to know complextity. Let's be more open for new ideas. I'm sure it's not as unuseful as you tend to consoder it. It might be fun for someone else to implement it. Just leave it for others.

[hvlad]

I see nor easy nor practical way to implement expression evaluator in gbak to skip restored rows according to the filter criteria. Such task is for the engine, not for gbak.

What @atronah asked for is much better and effective to implement using hand-maid partitioning, IMHO.

[aafemt]

On restore - yes. It is a little simpler on backup: you can add the condition to a query retrieving data from tables.