palantir / atlasdb

Transactional Distributed Database Layer
https://palantir.github.io/atlasdb/
Apache License 2.0
55 stars 14 forks source link

Products should be able to specify a set of tables to truncate #2933

Open hsaraogi opened 6 years ago

hsaraogi commented 6 years ago

This would allow cleaning up of cruft after migrations and unsupported tables.

sandorw commented 6 years ago

Possible impl skeleton:

Might be an interesting edge case around how we define the tables to be deleted, since we likely don't want them included in the schema definition at the point we want to drop them. We certainly don't want to recreate the tables after cleaning them up, for example.

tpetracca commented 6 years ago

Can we just infer them? Look at the schema provided in code, look at what tables exist in the keyspace, calculate the diff.

sandorw commented 6 years ago

Yes, that looks feasible. KVS.getAllTableNames fetches all non-KVS table names, including Atlas internal tables. Probably need to make sure that there's no cases where this deletes tables by accident, but I can't think of a good reason for removing a table from the schema without expecting it to be deleted.

tpetracca commented 6 years ago

There's probably some edges here around dynamically created tables in PG land that we probably need to protect against.

sandorw commented 6 years ago

Fair point, that's one argument for the more intentional version of explicitly listing tables.

clockfort commented 6 years ago

Also probably issues around migration rollback if we immediately truncate the unlisted table