treasure-data / chef-td-agent

Chef Cookbook for td-agent (Treasure Agent or Fluentd)
https://supermarket.chef.io/cookbooks/td-agent
Apache License 2.0
127 stars 121 forks source link

Support Internal Repos, Unit Testing, @Type, 3x-, and more #108

Closed jacknagz closed 6 years ago

jacknagz commented 6 years ago

to @yyuu

I have been working with this cookbook and figured I would contribute added features upstream! They are mainly to support the latest td-agent, and other changes to support unit and integration testing.

Changes

Testing

Locally with Chefpsec:

$ rspec  spec/unit/

td-agent::default
  converges without error
  creates td-agent.conf
  starts and enables the td-agent service

td-agent::install
  converges without error

Finished in 0.87151 seconds (files took 3.09 seconds to load)
4 examples, 0 failures

Also ran kitchen locally against 3x-ubuntu14.04 successfully.

jacknagz commented 6 years ago

Travis is failing to run the container for some reason, not sure if that's related to my changes or not

jacknagz commented 6 years ago

Another note: yum and apt no longer need to be dependencies of this cookbook, since those resources are now built into Chef. I can make the change if desired. As a result, this cookbook would be required to have Chef 12.14 as a minimum version.

yyuu commented 6 years ago

Travis is failing to run the container for some reason, not sure if that's related to my changes or not

It was totally not related to your changes. I confirmed that CI succeeded after refreshing cache on travis-ci :+1:

Another note: yum and apt no longer need to be dependencies of this cookbook, since those resources are now built into Chef. I can make the change if desired. As a result, this cookbook would be required to have Chef 12.14 as a minimum version.

For now it is left just for backward compatibility. Though I think that these extra dependencies must be harmless in general.

According to https://docs.chef.io/platforms.html, Chef Client 12.x will be EOL'd at April 30, 2018. Just in case, I want to wait until the date and then remove these deprecated configurations.

jacknagz commented 6 years ago

@yyuu added some more commits, please take a look!

Found an additional issue with one of the custom resources, where nested parameters were not being rendered as expected. I ended up consolidating the params_to_text helper into a library to share it among the three (filter, source, match).

I also added a fixture spec cookbook to verify everything's working well, I'd recommend adding chefspec tests to CI:

$ bundle exec rspec

td-agent::default
  converges without error
  creates td-agent.conf
  starts and enables the td-agent service

td-agent::install
  converges without error
  creates the td-agent group
  creates the td-agent user
  creates td-agent configuration directory
  creates td-agent runtime directory

td-agent-spec::filter
  converges without error
  creates unit test filter
  creates the filter config file

td-agent-spec::match
  converges without error
  creates unit test output
  creates the output config file

td-agent-spec::source
  converges without error
  creates unit test input
  creates the input config file

Finished in 2.34 seconds (files took 3.2 seconds to load)
17 examples, 0 failures

Thanks for the help so far on reviewing!

Jack

jacknagz commented 6 years ago

@yyuu that should do it, just pushed a commit to fix CI. pretty surprised the unit test didn't find that template error 🤔