Open pbchase opened 11 months ago
Related to this, REDCap might ask that older tables be altered to set ROW_FORMAT=DYNAMIC
. This post explains that each such table will be copied in the process: Some database tables need to be modified - RECOMMENDED This temporarily doubles their disk space usage until the old copy is purged.
There's a discussion of data preservation rules and the need to preserve log data for 7 years at How can we clear out large database tables logs?.
There's a discussion of the ramifications of projects with a very large number of log rows jamming up the pre-population of the users filter in the REDCap Logging query UI. There is a discussion about the value of defragging and rebuilding a log table with ALTER TABLE ``redcap_log_event7`` ENGINE = InnoDB;
See Logging page times out even after moving all log entries to redcap_log_event7
The redcap_log_event* tables could use some management tools.
These threads in the REDCap community could be helpful
Why are records left in redcap_log_event and redcap_log_view tables when a project is deleted? discusses what is and is not deleted from the log tables when a project is deleted. It appears to be inspiration for Sue Lowry's comments below about what she purges from the tables.
redcap_log_event too big Here are some helpful snippets:
Incomplete removal of data from redcap_log_event* on project delete provides more discussion on how REDCap overwrites log entries and strategies for doing that.
Migrating log entries from redcap_log_event describes how Andy Martin cleaned up their log event tables via a mix of data migration and indexing. It includes code for migration, adding the indices, measuring tables size, etc. Paul Litwin provides code for moving the log data in chunks when the log row count is very large.
It's unclear to me how the log migration work relates to the changes required to fix database tables that impeded Unicode support in older REDCap installations.