virocon-organization / viroconweb

A software to compute environmental contours. Django web-application package.
MIT License
2 stars 1 forks source link

Heroku can run out of memory unexpectedly #146

Closed ahaselsteiner closed 6 years ago

ahaselsteiner commented 6 years ago

I'm submitting a ...

Expected behavior

How should it work (with the bug fixed or the feature implemented)? Heroku should not run out of memory or at least inform the user about it.

Screenshot

Actual behavior

How does it currently work (with the bug causing problems or without the feature)? Heroku runs out of memory and then for example does not allow to open the user profile site anymore. It only shows a "server error 500 page".

Screenshot

Steps to reproduce the problem (how to see the actual behavior)

  1. In one session do a lot of calculations (measurement file, fitting, contours)
  2. Watch in the heroku logs how the memory exceeds 500 MB
  3. Click on "user profile" and receive the "error 500" page
ahaselsteiner commented 6 years ago

This is possible already solved since we now use with open and with urlopen. Before we had to manually close the file and we did not always do this. This might have caused the RAM to fill up.

image

ahaselsteiner commented 6 years ago

The problem is still apparent:

I did the following in one session: 10 times:

When trying to perform the 10th fit the application crashed: image

This is the log file showing the memory exceeding, an error R14 and then timing out very likely because of that: image

ahaselsteiner commented 6 years ago

We solved this by restricting the number of maximum requests: image

I did the 10 times data upload + fitting + contour to test this.

If I understand this correctly, after this number of maximum requests is reached the application restarts freeing up the memory. This causes a slight delay and can be seen in the heroku log: image

In the future we could try to increase the maximum number of requests. However, for now this is solved.