search

SumoLogic / sumologic-collector-chef-cookbook

Chef Cookbook for installing and configuring the SumoLogic collector for the Sumo Logic service
Other
30 stars 90 forks source link

Using chef/cookbook to update sumo.conf? #154

Open matttarantino opened 6 years ago

matttarantino commented 6 years ago

Howdy!

I recently discovered that some of our nodes are not reporting to Sumologic because they have a id/key that has expired. I went to update the data bag however after reading through the cookbook, it appears that the sumo.conf file is only generated when its first installed. Is there a method to updating the sumo.conf file using the cookbook that I am missing? It seems our alternative may be to write something custom, or update the id/key manually on each node.

Thanks!

stmarier commented 6 years ago

Just wanted to note I'm experiencing a similar issue

majormoses commented 6 years ago

I believe if you include https://github.com/SumoLogic/sumologic-collector-chef-cookbook/blob/master/recipes/sumoconf.rb it will update whenever there is a change.

Alternatively you can call the template directly from your wrapper if you prefer.

shubh2336 commented 5 years ago

@majormoses What's the best way to configure Sumo Collector to send logs to a new Sumologic account? I tried generating a new id/key pair & included sumoconf recipe. I could see that sumo.conf is updated with the new creds, but Sumo Collector is still sending logs to my old account. We're planning to shut down our old account & send all logs to a new sumo account going forward.

matttarantino commented 5 years ago

@shubh2336 I believe you need to restart the sumo collector process after making the change to the sumo.conf file in order for it to pick up the new credentials, otherwise it continues to use the old credentials.

Interestingly enough, the API will still accept data even if the old id/key pair has expired as long as the process is still running. Once the sumo collector process is restarted, it will not work with the existing id/key pair until it has been updated.

shubh2336 commented 5 years ago

@matttarantino Older key/id pair seem to be sticky in nature even after the restart. Sumo collector ends up ignoring the new keys & keep pushing logs to the account belonging to older key. Strangely, when i delete the existing collector via UI & then do a install_and_configure via chef, i could see logs appearing in new account.

majormoses commented 5 years ago

I took a closer look and ti sumo conf does not appear to restart the collector service. IMHO thats a bug with the cookbook and should be fairly easy to fix.

Interestingly enough, the API will still accept data even if the old id/key pair has expired as long as the process is still running. Once the sumo collector process is restarted, it will not work with the existing id/key pair until it has been updated.

Hmm thats scary, I am gonna ping some people at sumo to get more info on this it sounds like a vulnerability to me if authentication is not forced to be refreshed/brokered after some period of time after the collector starts.

matttarantino commented 5 years ago

@majormoses It was definitely strange. The credentials were unknowingly revoked but we only discovered it after some of our hosts had restarted and stopped reporting. The hosts that had not yet restarted were still reporting fine, even with bad creds. I am not sure of the version that we were running, but hopefully it has since been patched.