search

digitalmethodsinitiative / dmi-tcat

Digital Methods Initiative - Twitter Capture and Analysis Toolset
Apache License 2.0
367 stars 114 forks source link

capture/index.php calls upgrade with dryrun #347

Closed bosim closed 5 years ago

bosim commented 5 years ago

capture/index.php contains

$tests = upgrades(true);

which, even in dry_run mode, is costly. Especially the query below is very costly.

 $sql = "SELECT id, text FROM $v WHERE " .
                    "                LENGTH(text) > 140 AND " .
                    "                created_at >= '2017-11-01 00:00:00' AND " .
                    "                LENGTH(text) - LENGTH(REPLACE(text, '#', '')) > 0 " .
                    "                ORDER BY created_at ASC";

I wonder why the dry_run check is run everytime capture/index.php; just for convenience?

dentoir commented 5 years ago

Hi @bosim,

Yes, it is intentional, but if we want to check the necessity of upgrading to changes all the way back to 2014 it does takes too long particularly for the check you mention. We could run the test via the controller and cache the output, or limit the amount of tests to only the most recent steps. I've noticed the issue too and it caused issues with automatic monitoring of TCAT instances.

dentoir commented 5 years ago

For now, no warnings are displayed in the backend for upgrade steps which were added prior to May 2018 (in other words: none). Ofcourse upgrading using the cli method still works.

bosim commented 5 years ago

Nice change. Thanks for fixing.