search

csirtgadgets / massive-octo-spice

DEPRECATED - USE v3 (bearded-avenger)
https://github.com/csirtgadgets/bearded-avenger-deploymentkit/wiki
GNU Lesser General Public License v3.0
227 stars 62 forks source link

cif-smrt:always checking for router #489

Closed liweizhe closed 7 years ago

liweizhe commented 7 years ago

Hi, i have use cif for several days and it worked fine before,but it stop parsing feeds yesterday. i checked cif-smrt.log,parts of the log shows:

[2017-07-03T18:10:24,073Z][INFO]: checking for router... [2017-07-03T18:19:20,372Z][INFO]: checking for router... [2017-07-03T18:19:37,434Z][INFO]: checking for router... [2017-07-03T18:34:59,643Z][INFO]: checking for router... [2017-07-03T19:10:24,075Z][INFO]: checking for router... [2017-07-03T19:19:20,372Z][INFO]: checking for router...

but the cif-router is already running. in the cif-smrt log:

[2017-07-03T18:08:32,980Z][INFO]: staring up.. [2017-07-03T18:08:33,073Z][INFO]: started, waiting for messages.. [2017-07-04T08:58:43,395Z][INFO]: staring up..

I restarted the cif-services but it comes to the same situation.

liweizhe commented 7 years ago

in addition, the cif-smrt works fine using --testmode, but it stop parsing feeds automatically.

wesyoung commented 7 years ago

could be that cif-router or cif-starman is just "hung" ? did you reboot the whole box? cif-smrt tries to verify router is responding before it even tries to parse feeds, so if the http or zmq sockets aren't responding, it won't do anything...

liweizhe commented 7 years ago

Thanks for your reply, I am sure I have restarted both host and cif-services,but it didn't work. The cif-starman.log shows like:

[2017-07-05T10:25:37,987Z][INFO]: starting CIF::REST [2017-07-05T10:25:37,988Z][INFO]: starting CIF::REST [2017-07-05T10:27:37,999Z][INFO]: starting CIF::REST [2017-07-05T10:27:38,000Z][INFO]: starting CIF::REST [2017-07-05T10:27:38,042Z][INFO]: generating ping request... [2017-07-05T10:27:38,048Z][INFO]: sending ping... [2017-07-05T10:27:38,272Z][INFO]: starting CIF::REST [2017-07-05T10:29:38,013Z][INFO]: starting CIF::REST [2017-07-05T10:29:38,016Z][INFO]: starting CIF::REST [2017-07-05T10:31:38,025Z][INFO]: starting CIF::REST

Yesterday I try to use cmd "cif-smrt --testmode",after that the cif start parsing feeds automatically all day.But it stopped work at 0 am this morning.It seems there is trouble when cif try to make daily index in elasticsearch but I am not sure about that.And today it works well after I use testmode,besides,I did nothing with the cif conf before.

wesyoung commented 7 years ago

ah, check your elasticsearch cluster health. i wonder if it's YELLOW or RED because you're making a daily index instead of monthly (which is why we moved to monthly by default, elasticsearch is fast, but it has some architectural caveats). you may need to re-scale your cluster or delete some of the older indices (if i'm understanding your setup properly..).

you can also find more ways to verify this using the "Troubleshooting CIF" FAQ:

https://github.com/csirtgadgets/massive-octo-spice/wiki/FAQ https://github.com/csirtgadgets/massive-octo-spice/wiki/Troubleshooting-CIF

liweizhe commented 7 years ago

Thanks for your help, though I dont know exactly how it goes, the cif works fine after I backup es data and delete the older indices.

wesyoung commented 7 years ago

prob good to start here:

http://chrissimpson.co.uk/elasticsearch-yellow-cluster-status-explained.html