MTES-MCT / monitorfish

🎣 Improve control and monitoring of fishing vessels activities.
https://beta.gouv.fr/startups/monitorfish.html
GNU Affero General Public License v3.0
34 stars 7 forks source link

PSQLException: ERROR: out of shared memory #3907

Open louptheron opened 1 week ago

louptheron commented 1 week ago

Sentry Issue: MONITORFISH-14T

Exemple de requêtes : 1.

select date_time from positions where date_time < now() order by date_time desc limit 1]
  1. select lre1_0.trip_number,min(lre1_0.operation_datetime_utc),max(lre1_0.operation_datetime_utc) from logbook_reports lre1_0 where lre1_0.cfr=? and lre1_0.trip_number is not null and lre1_0.operation_type in ('DAT','COR') and lre1_0.operation_datetime_utc<=? and not(lre1_0.is_test_message) group by lre1_0.trip_number order by 2 desc fetch first ? rows only
  2. select operation_datetime_utc from logbook_reports where operation_datetime_utc < now() order by operation_datetime_utc desc limit 1

Tips de Vincent :

On a 550 chunks sur peu moins de 4 ans donc ~12 chunks par mois Donc je pense que quand on requête plus de 5-6 mois ça scan >64 chunks et ça fait l'erreur

Solution: Ajouter un where pour n'avoir que quelques mois (i.e 2)


PSQLException: ERROR: out of shared memory
  Hint: You might need to increase max_locks_per_transaction.
    at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2733)
    at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2420)
    at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:372)
    at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:517)
    at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:434)
...
(95 additional frame(s) were not displayed)

Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed: org.springframework.orm.jpa.JpaSystemException: JDBC exception executing SQL [select operation_datetime_utc from logbook_reports where operation_datetime_utc < now() order by operation_datetime_utc desc limit 1] [ERROR: out of shared memory
  Hint: You might need to increase max_locks_per_transaction.] [n/a]] with root cause
louptheron commented 1 day ago

@VincentAntoine J'ai modifié quelques requêtes (1 et 3) pour ne pas parcourir les chunks.

Par contre pour certaines nous sommes obligés de parcourir un grand nombre de chunks (i.e pour chercher toutes les marée d'un navire depuis 3 ans). J'ai cru voir que la propriété max_locks_per_transaction était plutôt petite par défaut, on pourrait l'augmenter un peu pour éviter ces erreurs, wdyt ?