Improve speed for set-open-durations and remove-os-info-quotes

[liZe]

These commands were retrieving the whole ShellAppIsOpen and OSVersion tables on the Python backend server, chunk by chunk. This was too slow for the production server data.

The new version proposed here still uses the ORM, but creates SQL-only requests that can be launched on the PostgreSQL server. Chunked queries, used to avoid the load of the whole tables in memory, are:

• useless because the data are not handled by Python, and
• impossible because SQLAlchemy (and I suppose PostgreSQL) cannot edit lines using the LIMIT keyword.

remove-os-info-quotes is now applied even on rows where quotes have already been removed. This no-op operation is probably as fast as filtering the lines before trimming the string. That is why the command no longer tells when no row has to be modified, and the related test has been changed accordingly.

Related to:

• https://phabricator.endlessm.com/T29220
• https://phabricator.endlessm.com/T29322

• 109

• 111

[adarnimrod]

BTW, the migrations ran in production. set-open-durations took 6 hours and 4 minutes and remove-os-info-quotes took just 7 minutes.