search

csirtgadgets / massive-octo-spice

DEPRECATED - USE v3 (bearded-avenger)
https://github.com/csirtgadgets/bearded-avenger-deploymentkit/wiki
GNU Lesser General Public License v3.0
227 stars 60 forks source link

Better ES status reporting through the cif api/client #389

Closed giovino closed 8 years ago

giovino commented 8 years ago

Scenario:

A CIF instance where ES has grown to 365 shards and is on a spinning disk. When ES or the box is restarted, it takes several minutes for ES to initialize itself. During that time the health check status is "red". 

$ curl -XGET 'http://localhost:9200/_cluster/health?pretty=true'
{
  "cluster_name" : "elasticsearch",
  "status" : "red",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 330,
  "active_shards" : 330,
  "relocating_shards" : 0,
  "initializing_shards" : 4,
  "unassigned_shards" : 396
}

it would be nice if cif-router would check for this and pass at along to the client. as it stands now the user sees this:

 cif -q example.com -d
[2016-02-24T09:55:24,443Z][INFO][main:268]: starting up client...
[2016-02-24T09:55:24,443Z][INFO][main:303]: running search...
[2016-02-24T09:55:24,443Z][DEBUG][CIF::SDK::Client:170]: uri created: https://localhost/observables?gzip=1&observable=example.com
[2016-02-24T09:55:24,443Z][DEBUG][CIF::SDK::Client:171]: making request...
[2016-02-24T09:55:54,737Z][INFO][CIF::SDK::Client:175]: status: 422
[2016-02-24T09:55:54,738Z][INFO][CIF::SDK::Client:181]: response size: < 1MB
[2016-02-24T09:55:54,738Z][DEBUG][CIF::SDK::Client:184]: decoding content..
[2016-02-24T09:55:54,738Z][DEBUG][CIF::SDK::Client:187]: decompressing...
[2016-02-24T09:55:54,738Z][DEBUG][CIF::SDK::Client:193]: Data input to gunzip is not in gzip format at /usr/local/share/perl/5.18.2/CIF/SDK/Client.pm line 189.
invalid request at /usr/local/bin/cif line 310.

status 422 & "invalid request"

you could imagine a better response from the CIF API:

 cif -q example.com -d
[2016-02-24T09:55:24,443Z][INFO][main:268]: starting up client...
[2016-02-24T09:55:24,443Z][INFO][main:303]: running search...
[2016-02-24T09:55:24,443Z][DEBUG][CIF::SDK::Client:170]: uri created: https://localhost/observables?gzip=1&observable=example.com
[2016-02-24T09:55:24,443Z][DEBUG][CIF::SDK::Client:171]: making request...
[2016-02-24T09:55:54,737Z][INFO][CIF::SDK::Client:175]: status: 503 OR 504
[2016-02-24T09:55:54,738Z][INFO][CIF::SDK::Client:181]: response size: < 1MB
[2016-02-24T09:55:54,738Z][DEBUG][CIF::SDK::Client:184]: decoding content..
[2016-02-24T09:55:54,738Z][DEBUG][CIF::SDK::Client:187]: decompressing...
[2016-02-24T09:55:54,738Z][DEBUG][CIF::SDK::Client:193]: Data input to gunzip is not in gzip format at /usr/local/share/perl/5.18.2/CIF/SDK/Client.pm line 189.
Error: The ElasticSearch cluster is reporting a status of "Red", investigate the health status of the ElasticSearch cluster.