[MetaTicket] Improve preview-generator performances

[inkhey]

About

Preview generator code as many performances issues, especially with big spreedsheat file. Here are some proposal to fix this

Proposals :

• [X] 1.generate a lockfile to preview multiple concurrent same file generation.(https://github.com/algoo/preview-generator/issues/187)
• [ ] 2.remove specific case for libreoffice/scribus leading to generate 2 time the same PDF when asking for a JPEG.
• [ ] 3. RAM limit/ CPU time configuration : home-made dans preview generator (via https://stackoverflow.com/a/1689991)-
• [ ] 4. solve issue with big spreedsheet (see specific section)
• [ ] 5. At higher level (not intor preview-generator code itself, in web app using preview-generator) : find a way to kill process if during too long, may be nice to document this a bit in preview-generator documentation, if such solution is found.
• [X] 6. use lockfile instead of threading.lock() for libreoffice lock to prevent concurrent libreoffice work (which will fail) (https://github.com/algoo/preview-generator/issues/187)

About point 4 : big spreedsheet.

• some old version of libreoffice (6.0.7.3) fail to generate pdf preview of big spreedsheat in a correct amount of time (>10 minutes) whereas newer one work correcty (6.4.2.2 took few seconds).
• (in french) dans la configuration actuelle, le fichier généré fait 4000 pages. J'ai un peu regardé mais j'ai pas trouvé d'option magique pour permettre de rendre ça plus lisible ou plus léger dans tout les cas de figure. Il y à pas mal de configuration pour limiter à un nombre fixe de page, de changer la taille du fichier de sortie (A3 , portrait), etc… ça serait intéressant de savoir si ces options sont conservé ou non dans les fichiers tableurs eux-même, autrement ça n'à pas forcément de sens de surchargé la configuration par défaut pour une configuration qui marche "un peu mieux".
• (in french) l'export de la feuille de calcul unique paraît intéressante : https://www.collaboraoffice.com/release-news/full_spreadsheet_pdf_export/ mais dans les faits sur des trop gros fichiers, cela crée des pdf illisible car trop gros (ni le lecteur de pdf de gnome ni gimp n'accepte celui généré par le gros fichiers qui plante tracim).
  • (in french) HS: il est possible de transformer le PDF en PDF normalisé archive, étant donné qu'on parle là de format "pivot", uniquement pour de la génération de preview, c'est peut être intéressant (j'ai pas creusé plus)
• Another solution may be to have distinct behavior for "big" spreedsheat, issue here is how to distinguist a "big spreedsheat" ?

[inkhey]

For Libreoffice lock(6), until we do decide to use a server-based mecanism (see https://github.com/algoo/preview-generator/issues/77), there is a balance between allowing multiple parallel libreoffice and avoiding creating too much config data : libreoffice require to set env dir for running and create data in this place, if 2 process try to use same env at the same time, it mean like opening multiple file with same libreoffice instance, preview generator cannot deal correctly with this.

I do see 2 different solution for safe mecanism here:

• One libreoffice instance per computer at the same time using filelock with a fixed named save on cache_path dir.
• One libreoffice per process id (os.getpid()) or per thread id using or https://docs.python.org/3/library/threading.html#threading.get_native_id . This mean we can keep current lock mecanism (for process lock) or completely remove the need of lock here.

Edited: solution choosen by #187 is one libreoffice instance per cache_dir.

[inkhey]

About point 3, i created a poc: https://github.com/algoo/preview-generator/commit/6179621f97ed6971e102dd1984fc77739c36a440

Idea is to use always our own method based on modern subprocess.run method everywhere instead of check_call and check ouput old method. This way we can adjust with default limitation for all subprocess used.

What's need clarification:

• Where does we put limitation value. I first think about making limitation builder parameter but this does not work as some usage of subprocess method are done in classmethod.
• type of limitation to choose (check poc)
• value of limitation