search

Call-for-Code-for-Racial-Justice / Legit-Info

"Legit Info" app helps residents understand the impact of legislation and policies based on their country, state, county, city, district location
Apache License 2.0
67 stars 74 forks source link

gunicorn failing on searching for laws #88

Closed upkarlidder closed 3 years ago

upkarlidder commented 3 years ago

Error 502 in browser.

Recent change that could be causing this: #87

Server logs:

Jun 16 14:10:17 Code Engine legit-info-app-00009-deployment-788fcdb465-pfr4b error reverse proxying request; sockstat: sockets: used 1786
TCP: inuse 4 orphan 1 tw 9 alloc 241 mem 69
UDP: inuse 0 mem 5
UDPLITE: inuse 0
RAW: inuse 0
FRAG: inuse 0 memory 0
Jun 16 14:10:27 Code Engine legit-info-app-00009-deployment-788fcdb465-pfr4b error reverse proxying request; sockstat: sockets: used 1784
TCP: inuse 5 orphan 1 tw 9 alloc 243 mem 78
UDP: inuse 0 mem 5
UDPLITE: inuse 0
RAW: inuse 0
FRAG: inuse 0 memory 0
Jun 18 07:02:51 Code Engine legit-info-app-00009-deployment-788fcdb465-pfr4b 06-18 14:02:49 views:index [INFO] 195: username=cfcadmin
Jun 18 07:04:20 Code Engine legit-info-app-00009-deployment-788fcdb465-pfr4b +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:10)
Jun 18 07:04:20 Code Engine legit-info-app-00009-deployment-788fcdb465-pfr4b +0000] [10] [INFO] Worker exiting (pid: 10)
Jun 18 07:04:20 Code Engine legit-info-app-00009-deployment-788fcdb465-pfr4b error reverse proxying request; sockstat: sockets: used 2832
TCP: inuse 5 orphan 3 tw 16 alloc 344 mem 131
UDP: inuse 0 mem 4
UDPLITE: inuse 0
RAW: inuse 0
FRAG: inuse 0 memory 0
Jun 18 07:04:20 Code Engine legit-info-app-00009-deployment-788fcdb465-pfr4b +0000] [11] [INFO] Booting worker with pid: 11
Jun 18 07:04:22 Code Engine legit-info-app-00009-deployment-788fcdb465-pfr4b 06-18 14:03:42 views:index [INFO] 195: username=cfcadmin
Jun 18 07:04:30 Code Engine legit-info-app-00009-deployment-788fcdb465-pfr4b **Using PostgreSQL**
Jun 18 07:05:50 Code Engine legit-info-app-00009-deployment-788fcdb465-pfr4b +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:11)
Jun 18 07:05:50 Code Engine legit-info-app-00009-deployment-788fcdb465-pfr4b +0000] [11] [INFO] Worker exiting (pid: 11)
Jun 18 07:05:50 Code Engine legit-info-app-00009-deployment-788fcdb465-pfr4b error reverse proxying request; sockstat: sockets: used 2831
TCP: inuse 5 orphan 0 tw 7 alloc 338 mem 98
UDP: inuse 0 mem 5
UDPLITE: inuse 0
RAW: inuse 0
FRAG: inuse 0 memory 0
Jun 18 07:05:50 Code Engine legit-info-app-00009-deployment-788fcdb465-pfr4b +0000] [12] [INFO] Booting worker with pid: 12
Jun 18 07:05:50 Code Engine legit-info-app-00009-deployment-788fcdb465-pfr4b **Using PostgreSQL**
Jun 18 07:06:10 Code Engine legit-info-app-00009-deployment-788fcdb465-pfr4b error reverse proxying request; sockstat: sockets: used 2830
TCP: inuse 6 orphan 5 tw 7 alloc 342 mem 100
UDP: inuse 0 mem 4
UDPLITE: inuse 0
RAW: inuse 0
FRAG: inuse 0 memory 0
upkarlidder commented 3 years ago

The def results(request, search_id): method calls make_csv(search_id, laws_list) to create the CSV file.

https://github.com/Call-for-Code-for-Racial-Justice/Legit-Info/blob/965a37502f6b559240c8b3f07e78f77ac1ee5853/cfc_app/views.py#L270

I believe this causes the 502 reported above if there are a lot of results and the resulting CSV file is big. I commented out make_csv call and the search works now.

  1. recommend that we create the CSV file when the user needs it and not when searching for legislation.
  2. recommend the CSV creation happens in a different thread or CE job so the UI is not blocked. Need further discussion on this.
github-actions[bot] commented 3 years ago

:wave: Hi! This issue has been marked stale due to inactivity. If no further activity occurs, it will automatically be closed.