search

CottageLabs / willow

Willow is an implementation of the Fedora/Samvera stack by Cottage Labs. It is built with Docker containers, which simplify development and deployment onto live services.
http://willow.cottagelabs.com/
6 stars 2 forks source link

Hardware resource and application monitoring #66

Open Nimphal opened 7 years ago

emanuil-tolev commented 7 years ago

I'm not sure Newrelic is the best option actually, but for now we can go with it if I can't find better ones.

It's a great product, but I am disappointed with the pricing (I don't want to buy into everything for hundreds of dollars / month), and it doesn't work at all with DOAJ for example - no obvious reason for the moment. Competitors have emerged in the last several years.

martyn-w commented 7 years ago

Have a look at monitorix: http://www.monitorix.org/ we are experimenting with it as part of Webber.

emanuil-tolev commented 7 years ago

That looks good, but Docker monitoring may require some specific tools, especially around service discovery (what's running). Since docker cloud manages our infrastructure we have to assume the servers could be destroyed and recreated at any time.

We do need traditional stuff as well (disk % full, CPU, RAM, network of the server itself) but I was planning to see if there are Docker-specific features that some tools might offer. SaaS is also generally plus since we don't have to then host an interface for the tool in addition to the app, but I recognise that the fact your data is only on the SaaS servers is actually the problem with Newrelic and their pricing.

emanuil-tolev commented 7 years ago

emanuil_tolev [2:28 PM] an interesting monitoring service: http://www.netuitive.com/pricing/ (it's $18/mo per host, with up to 10 docker containers on that host, and $15/mo if billed annually).

pros: 12 months metrics retention, don't need to set up our own metrics for the various services and hardware resources. Uses weak AI to detect anomalies (e.g. resource use spikes, response time spikes), so hopefully less setup for us saying "this metric below 1.2 means alarm".

cons: not free, we won't have our own monitoring setup and own the data (but I'd argue we have a lot of other work besides setting up federated monitoring stacks)

risks: it really needs to have in-depth data collection inside docker, and from common software (postgres, solr, redis metrics) + the ability to measure web app and background job metrics. http://www.netuitive.com/integrations/ notes integration with linux (for hardware resource metrics?), nginx, ruby, postgres, apache, solr + zookeeper (= distributed solr). So looks reasonable to try at least.

[2:31]
Personally, I think it looks great compared to Newrelic on pricing, and basic solutions such as per-host logging with open-source software take a lot of effort to get to good federated stats. In lieu of that effort, they ask for $15-18 per host per month. (edited)

Free trial for 21 days. We'll try it and make a decision within the scope of this issue.

Nimphal commented 7 years ago

Has there been any movement in this front?