script for status and restarting VCell services - Githubissues

virtualcell / vcell

Virtual Cell Framework

http://vcell.org/

Other

66 stars 20 forks source link

script for status and restarting VCell services #1051

Closed jcschaff closed 9 months ago

jcschaff commented 9 months ago

Scripts (run as user vcell on vcell servers)

vcell-services - new script to check vcell swarm cluster status and restart services

BetterStack

monitor configuration monitors simulation and login for release and alpha sites (see https://uptime.betterstack.com/team/44952/monitors)
Slack integration slack channel vcell-monitor on 'UCHC-HPC'
status page https://status.vcell.org

Elastalert

monitors vcell logs ElastAlert is installed on elk and queries ElasticSearch for out of memory issues on vcell services which are not always visible using BetterStack alerts.
Slack integration alerts are sent to slack channel vcell-alerts on UCHC-HPC.

jcschaff commented 9 months ago

run vcell-services as user vcell on a vcell server node

jcschaff commented 9 months ago

currently this bash script has been manually installed at ~vcell/.local/bin/vcell-services

TODO: needs to be added to the GitHub CD scripts to ensure that script updates are deployed.