feature: Significant performance boost for filer directory listings

Description

When removing mptt #1351 once concern were database hits. While getting the full path of a folder with and without mptt requires n db hits, n being the nesting of the current folder typically is small.

For directories containing m files, the performance bottleneck appeared to be getting the thumbnails requiring m database hits. Filer limits the number of files per page to 20 to avoid this problem to take excessive time.

This PR builds upon #1351 and improves directory listings in filer by

Annotating the expected paths of a file's thumbnail URLs (regular, and double resolution). That's a bit tricky and requires Django subqueries. It gives a great heuristic for checking if a relevant thumbnail already exists. If so, it is used without having easy thumbnails, hit the db again to check.
Avoid creating missing thumbnails during the directory request. Instead, an additional admin URL generates missing thumbnails in a separate request each (potentially taking advantage of multiple worker instances).
Use select_related on a folder's and file's owner. This was a classical m+1 issue with the User object.
Cache template tag file_icon_url which is called several times per file when rendering a directory. Also, it needlessly checked for the existence of thumbnails in the db.
For getting thumbnails, do not check if the original file exists, if not on the FileSystemStorage: Otherwise this tests triggers network connections to the file storage (such as AWS) for each file in the directory.

Accompanying changes are:

I have set the default pagination limit from 20 to 100.
Thumbnails are loaded lazily to balance the load a little.

Further performance improvement lever:

Configure and use easy-thumbnail's celery integration to have necessary thumbnails always at hand

Performance impact

This is just one example: 71 files at nesting level 2 requires

DB hits when ...	Current filer	This PR
Listing directory	301	13
Selecting file	952	13

DB queries are independent of folder size (in terms of number of files/folders in that folder) as opposed to between 4m and 13m.

Way forward

This performance improvement is so significant that I suggest dropping the mptt dependency and merging the request for version 3.0 even before any major redesign for filer (see #1348). (In a separate upcoming PR, I would like to address minor styling issues, which might also go into 2.3).

Related resources

1351
...

Checklist

[x] I have opened this pull request against master
[x] I have added or modified the tests when changing logic
[x] I have followed the conventional commits guidelines to add meaningful information into the changelog
[x] I have read the contribution guidelines and I have joined #workgroup-pr-review on Slack to find a “pr review buddy” who is going to review my pull request.

Codecov Report

Merging #1353 (5247c38) into master (e920e55) will increase coverage by 0.13%. The diff coverage is 82.55%.

@@            Coverage Diff             @@
##           master    #1353      +/-   ##
==========================================
+ Coverage   72.41%   72.54%   +0.13%     
==========================================
  Files          72       73       +1     
  Lines        3284     3311      +27     
  Branches      534      538       +4     
==========================================
+ Hits         2378     2402      +24     
- Misses        739      742       +3     
  Partials      167      167

Impacted Files	Coverage Δ
filer/thumbnail_processors.py	`38.46% <ø> (ø)`
filer/admin/fileadmin.py	`78.75% <52.63%> (-8.75%)`	:arrow_down:
filer/templatetags/filer_admin_tags.py	`88.17% <86.36%> (-0.44%)`	:arrow_down:
filer/admin/folderadmin.py	`72.81% <87.50%> (+0.87%)`	:arrow_up:
...der_index_together_remove_folder_level_and_more.py	`100.00% <100.00%> (ø)`
filer/models/filemodels.py	`85.58% <100.00%> (-0.07%)`	:arrow_down:
filer/models/foldermodels.py	`86.36% <100.00%> (+0.37%)`	:arrow_up:
filer/settings.py	`79.38% <100.00%> (+0.65%)`	:arrow_up:

django-cms / django-filer