Closed sbearcsiro closed 3 years ago
Checking whether there is already an image with the same URL is slow. In the postgres logs with slow query logging enabled and using a prod db copy we see many entries like the following during a large batch upload:
2021-05-28 11:31:50.501 AEST [5205] images@images-prod DETAIL: parameters: $1 = 'https://inaturalist-open-data.s3.amazonaws.com/photos/72540103/original.jpeg?1589331696', $2 = '{https://inaturalist-open-data.s3.amazonaws.com/photos/72540103/original.jpeg?1589331696}'```
Checking whether there is already an image with the same URL is slow. In the postgres logs with slow query logging enabled and using a prod db copy we see many entries like the following during a large batch upload: