page size is not constrained in searching images

[sadeghim]

You can make URLs like https://images.ala.org.au/?q=&fq=&offset=5000&max=1000&sort=dateUploaded&order=desc which will cause some load on the server. Though the rate limiter of nginx will kick in and some of the images will fail to load.

[djtfmartin]

Theres a max offset currently set to 1000.

This is configurable:

https://github.com/AtlasOfLivingAustralia/image-service/blob/master/grails-app/conf/application.yml#L268

I thought 10k was reasonable without causing performance issues.

@sadeghim did you have an idea of what the max offset should be?

[sadeghim]

nginx has the burst limit of 800 and I see errors when the max is 1000: *671972 limiting requests, excess: 800.976 by zone "nginxratelimitzone",

I tested with 500 and could hit that again. But 200 was completely fine and I think reasonable.

[djtfmartin]

Ive deployed some changes to images-dev with some additional config settings

https://github.com/AtlasOfLivingAustralia/image-service/blob/develop/grails-app/conf/application.yml#L269

elasticsearch:
  indexName: "images"
  host: "127.0.0.1"
  scheme: "http"
  port1: 9200
  port2: 9201
  maxFacetSize: 10000
  maxOffset: 10000
  maxPageSize: 200
  defaultPageSize: 10