ELK to store and analyze PC2 web service logs

PathwayCommons / cpath2

Biological pathway data integration and access platform (Pathway Commons)

http://www.pathwaycommons.org/pc2/

MIT License

6 stars 5 forks source link

ELK to store and analyze PC2 web service logs #233

Closed IgorRodchenkov closed 7 years ago

IgorRodchenkov commented 8 years ago

Use Elastisearch, Logstash, Kibana stack (ELK) for PC2 logs instead of JPA/H2 database and current /log views and scripts.

(Might we'll be able to do other interesting things in the future, such as to replace current BioPAX full-text index/search sub-system in cPath2, or even store BioPAX (as JSONP) in Elasticsearch).

IgorRodchenkov commented 8 years ago

Also, current /log db/services becomes useless if we are to have several clones of PC2 behind a load balancer (e.g., NGINX).

IgorRodchenkov commented 7 years ago

Ok, starting PC9, we are going to use Elasticsearch, Filebeat(s), Kibana for log analysis and visualisations, and remove the internal db table 'logentity' and related code.