[BUG] Limit of 500 people affects functionality

The bug

PR #3069 introduced a limit of 500 people for performance reasons. However, it's affecting functionality:

I can't find some people anywhere but by opening a photo and going to the info panel
I can't merge a face with another when the face is beyond that 500 limit (that is, those with few photos since the query is ordered by count of assets DESC)

The merge functionality is the most affected by this really: in my case I have two people having like 1 or 2 photos linked to them; I want to merge them together but I can't because when I try to start the merge procedure from one face, I can't find the other.

For the moment I'm working around it using the mergePerson API, but I believe some sort of server-side pagination should be introduced maybe.

The OS that Immich Server is running on

Docker on Debian 11

Version of Immich Server

v1.68.0

Version of Immich Mobile App

v1.68.0

Platform with the issue

[X] Server
[ ] Web
[ ] Mobile

Your docker-compose.yml content

https://github.com/immich-app/immich/releases/download/v1.68.0/docker-compose.yml

Your .env content

# comments stripped
DB_HOSTNAME=immich_postgres
DB_USERNAME=postgres
DB_PASSWORD=[redacted]
DB_DATABASE_NAME=immich
REDIS_HOSTNAME=immich_redis
UPLOAD_LOCATION=[redacted]
TYPESENSE_API_KEY=[redacted]
IMMICH_WEB_URL=http://immich-web:3000
IMMICH_SERVER_URL=http://immich-server:3001
IMMICH_MACHINE_LEARNING_URL=http://immich-machine-learning:3003
IMMICH_API_URL_EXTERNAL=[redacted]

Reproduction steps

1. take a library where you have more than 500 recognized people
2. some of those people will have few photos linked to it (like 1 or 2, probably mistakes by the face recognition AI)
3. go to the "all people" page of the web app and you won't find those pople
4. try to merge a person you do find with another one from point 2: you won't find the latter

Additional information

No response

immich-app / immich