search

pimbongaerts / mesophotic

Ruby on Rails application to host scientific information on mesophotic coral ecosystems
http://www.mesophotic.org
MIT License
1 stars 1 forks source link

Is Mesophotic.org down? #106

Closed AIHAgreda closed 3 years ago

AIHAgreda commented 3 years ago

Hi @pimbongaerts and @ryanbooker - it looks like the page is down again Screen Shot 2021-01-20 at 9 01 01 AM

ryanbooker commented 3 years ago

Seems to be ok now…

pimbongaerts commented 3 years ago

I restarted the VPS. Not sure why it keeps happening though?

On Thu, Jan 21, 2021, 11:58 AM Ryan Booker notifications@github.com wrote:

Seems to be ok now…

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pimbongaerts/mesophotic/issues/106#issuecomment-764008236, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFTT2GEB47JUYXHEOE7FX3S25N2PANCNFSM4WK7RVJQ .

ryanbooker commented 3 years ago

Not sure. Seems like a questions for Dreamhost… I'll ask them. Is this the second time we've had to do that?

pimbongaerts commented 3 years ago

Sounds good! I think the third or fourth time.

On Thu, 21 Jan 2021 at 12:39, Ryan Booker notifications@github.com wrote:

Not sure. Seems like a questions for Dreamhost… I'll ask them. Is this the second time we've had to do that?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pimbongaerts/mesophotic/issues/106#issuecomment-764029993, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFTT2CSCRYVPJB4JHISUHDS25ST7ANCNFSM4WK7RVJQ .

ryanbooker commented 3 years ago

Ticket lodged

ryanbooker commented 3 years ago

They replied with basically: We can't see anything wrong, or any errors in settings. Let us know if it keeps happening.

veronicar239 commented 3 years ago

Seems like some select pages such as the About page are down? But the Publications page and Photos page are working fine.

http://mesophotic.org/about

Screen Shot 2021-01-27 at 10 57 35 am
ryanbooker commented 3 years ago

FYI. I had to reboot the VPS again this afternoon. So I have logged another ticket.

ryanbooker commented 3 years ago

Dreamhost replied with some useful information. They said the site keeps hitting a memory limit. So we need to look into that… I've added "New Relic" monitoring. I've used it before on another site. It gives some useful metrics re memory usage and speed etc.

pimbongaerts commented 3 years ago

Site just went down again, so restarted VPS

ryanbooker commented 3 years ago

I've made several changes that I hope will reduce the memory footprint of several pages. Let's see… :)

ryanbooker commented 3 years ago

Note to self: Problem seems mostly related to

AIHAgreda commented 3 years ago

Hi @ryanbooker and @pimbongaerts - it happened again

Screen Shot 2021-02-18 at 10 46 32 AM
pimbongaerts commented 3 years ago

VPS restarted. @ryanbooker - again related to memory footprint?

ryanbooker commented 3 years ago

I assume so. But I'm not sure what. It looks like it died just before midnight last night, and memory usage then seems to have been only averaging around 300MB, well under the VPS memory (perhaps there is something that temporarily spikes memory), and I've spent the last week or more, reducing memory usage in some key places, and speeding several things up in the process…

I thought it was keywords/wordcloud generation as that was a hugely slow and memory inefficient process. I've already done a lot to speed that up, and have improved it significantly. I am also trying to work out how to move it into the DB, instead of Ruby, which should be a lot faster and more memory efficient.

I also wonder if the recent updates they did to the VPS have made an impact, because this problem is new.

We have also move to ActiveStorage for images recently, instead of Paperclip after that stopped working. So it could be related to that as well.

pimbongaerts commented 3 years ago

Thanks so much for looking into it Ryan!

veronicar239 commented 3 years ago

Hi @ryanbooker , I just added 6 new photos and while they show on the Images page (http://www.mesophotic.org/photos) and in the specific Behind the Science post (http://www.mesophotic.org/posts/68), the individual photo pages show error messages. For example, http://www.mesophotic.org/photos/420 shows the error log message "We're sorry, but something went wrong."

However, other (older) individual photo pages (such as, http://www.mesophotic.org/photos/419) open just fine.

ryanbooker commented 3 years ago

I think I've solved this memory issue, at least enough for the moment. Closing this issue. :)

veronicar239 commented 3 years ago

Sorry to re-open this @ryanbooker but the same initial error message is showing again!

Screen Shot 2021-04-02 at 12 29 15 pm
pimbongaerts commented 3 years ago

Just restarted VPS in the meantime