WordPress / openverse

Openverse is a search engine for openly-licensed media. This monorepo includes all application code.
https://openverse.org
MIT License
254 stars 203 forks source link

Django's image admin view times out #5174

Open krysal opened 2 days ago

krysal commented 2 days ago

Description

The /admin/api/image/ view doesn't load with production data. You can confirm this by trying to access in staging (duplicates production data) if you have access: https://api-staging.openverse.org/admin/api/image/

I wondered if this was caused due to a heavy DB query and got what was being executed locally (see below). I used the same query in Grafana against the staging and production databases, and the result was returned immediately, so a problem at the DB level was discarded.

query ```sql SELECT "image"."id", "image"."created_on", "image"."updated_on", "image"."identifier", "image"."foreign_identifier", "image"."title", "image"."foreign_landing_url", "image"."creator", "image"."creator_url", "image"."thumbnail", "image"."provider", "image"."url", "image"."filesize", "image"."filetype", "image"."watermarked", "image"."license", "image"."license_version", "image"."source", "image"."last_synced_with_source", "image"."removed_from_source", "image"."view_count", "image"."tags", "image"."category", "image"."meta_data", "image"."width", "image"."height", COUNT("nsfw_reports"."id") AS "total_report_count", (COUNT("nsfw_reports"."id") - COUNT("nsfw_reports"."decision_id")) AS "pending_report_count", MIN("nsfw_reports"."created_at") AS "oldest_report_date" FROM "image" INNER JOIN "nsfw_reports" ON ("image"."identifier" = "nsfw_reports"."identifier") WHERE "nsfw_reports"."id" IS NOT NULL GROUP BY "image"."id" HAVING (COUNT("nsfw_reports"."id") - COUNT("nsfw_reports"."decision_id")) > 0 ORDER BY 27 DESC, 28 DESC, 29 ASC, "image"."id" DESC; ```

The query returns a count of 458, which the admin might be trying to load all at once, causing a rendering issue. Instead, the audio view (admin/api/audio/), with only 38 audio tracks reported, loads fine. We should limit the number of media items shown on these pages and paginate them.

Reproduction

  1. Login into https://api-staging.openverse.org/admin
  2. Go to the view https://api-staging.openverse.org/admin/api/image/
  3. See error.

Screenshots

The image view when trying to access it.

The audio view loading fine.