search

ioos / comt

IOOS Coastal and Ocean Modeling Testbed. See:
https://github.com/ioos/comt/wiki
MIT License
1 stars 7 forks source link

comt THREDDS server down #1

Closed rsignell-usgs closed 10 years ago

rsignell-usgs commented 10 years ago

the comt THREDDS server is down http://comt.sura.org/thredds

brianmckenna commented 10 years ago

THREDDS was reported down, actions taken

1) checked the web server status to determine if this was just THREDDS or server/Apache

$ curl -Is http://comt.sura.org | head -n 1 | cut -d$' ' -f2

returns

200

server returned HTTP 200 indicating both the server and Apache were properly functioning, issue must exist with THREDDS or the proxy between Apache/THREDDS

2) log in to comt.sura.org, check if THREDDS process is running

$ ps ax | grep java | grep -i catalina

should return (or similar)

19414 ?        Sl     1:39 /usr/java/jdk1.7.0_17/bin/java <...> org.apache.catalina.startup.Bootstrap start

in this case, there was no THREDDS process listed indicating at least part of the problem was that THREDDS was no longer running (worth checking /var/www/thredds_instance/logs/catalina.out for any indication of why)

3) before restarting, verify all necessary mount points are mounted

$ df

shows all active mounts, check these active mounts against /etc/fstab

In this case, the /data mount was not listed, indicating it was no longer mounted. Network issues or remote server reboot are among possibilities, without involving IT, we can simply try and remount:

mount -a for all or mount /data for the one we've identified as missing

mount was reattached, so any upstream problem was likely already solved.

4) restart THREDDS

/etc/init.d/tomcat_thredds restart

THREDDS was operating properly at this point.

rsignell-usgs commented 10 years ago

@brianmckenna: Awesome. Thanks both for the fish and for the fishing technique!