PurdueElectricRacing / FullSend

Automated email service for the Purdue Electric Racing team.
https://full-send.herokuapp.com
0 stars 1 forks source link

Server Error: Too many connections for "fuztyzcqipudmu" #33

Open RyanSchw opened 5 years ago

RyanSchw commented 5 years ago

Grabbed from Sentry: https://sentry.io/share/issue/546d505def0e4686ad03ddea18f27422/ fuztyzcqipudmu is the name of the user when getting the ServerAuthentication object from the database.

I think this issue is due to the fact that the scheduler has multiple jobs running at the same time, when only one is supposed to be running concurrently.

scheduler.add_job(refreshtoken, 'cron', minute='50', jitter=240)

Does not have max_instances set, therefore it is calling to update the token multiple times (at the same time).

See update log -- notice how upwards of 6 "update" events are sent in an hour (expected 1-2): https://sentry.io/share/issue/695f8d83db8f405cba5484a87610a66e/

RyanSchw commented 5 years ago

Since this schedule is stored in memory, causes some issues where the memory isn't being released properly and multiple scheduled instances run at the same time.

Need to remove this scheduling and just refresh the token if access_token is expired. (relevant link?)

RyanSchw commented 5 years ago

Temporary fix: heroku restart --app full-send will restart the connections. Issue comes back after about 24 hours.