search

digitalfabrik / integreat-cms

Simplified content management back end for the Integreat App - a multilingual information platform for newcomers
https://digitalfabrik.github.io/integreat-cms/
Apache License 2.0
55 stars 33 forks source link

Link list is loading very slow in regions with a lot of content #2539

Closed timobrembeck closed 2 days ago

timobrembeck commented 8 months ago

Describe the Bug

Steps to Reproduce

  1. Go to https://cms-test.integreat-app.de/muenchen/analytics/linkcheck/invalid/
  2. Wait for 20s
  3. See page

Expected Behavior

The linkcheck list should load in a couple of seconds (maybe a maximum of 5s for really large regions?)

Actual Behavior

It's considerably slower

Additional Information

After this is fixed, the linkcheck row can be shown again in the dashboard:

Revert https://github.com/digitalfabrik/integreat-cms/pull/2538/commits/1f9be5876b79710490580d88800e0c3377b688e7

david-venhoff commented 6 months ago

I tried to reproduce this by creating one page with ~750 different valid urls and another page with ~750 different invalid urls. Then I made 32 copies of each page.

However, the relevant code of the dashboard ran in about 300-500 ms for me. I also noticed that the example link (https://cms-test.integreat-app.de/muenchen/analytics/linkcheck/invalid/) loads in about 7 seconds, which seems a lot better than 20 seconds. Could it be that this issue has already been fixed?

Otherwise I could imagine that the problem only happens when there are lots of pages that contain different links, but that is hard to test locally.

Apart from that, one simple way to improve the performance for the dashboard would be to specify prefetch_content_objects=False in the call to filter_urls. This roughly halves the time required on my pc.

david-venhoff commented 4 months ago

Adding the blocked label because it is hard to reproduce locally without #2516

osmers commented 1 month ago

Is this maybe an infrastructure topic/problem as well?