Open luck02 opened 9 years ago
Hi there --
The best examples are in the test suites with test-kitchen:
suites:
- name: default # server
run_list:
- recipe[java::default]
- recipe[elkstack::default]
- name: agent # java agent with server
run_list:
- recipe[java::default]
- recipe[elkstack::agent]
- name: forwarder # alternative golang agent with server
data_bags_path: "test/integration/forwarder/data_bags"
encrypted_data_bag_secret_key_path: "test/integration/forwarder/encrypted_data_bag_secret"
run_list:
- recipe[java::default]
- recipe[elkstack::default]
- recipe[elkstack::forwarder]
It shouldn't require any special setup for the most common use case. We'll work on a blog post or something similar that could elaborate on it. Thanks!
Hey @martinb3 , thanks for the feedback!
So I'm in the same boat as @luck02 a bit, in that I am coming at this cookbook with completely fresh eyes, trying to figure out how to use it, and coming up with some questions. I'm going to pile on here, if that's cool, in the hopes that some of this stuff will be bundled up to the README.md or a blog post. I am happy to submit a PR for the former, btw, if that is desirable.
Anyways, I am trying to get this up in Test Kitchen to poke around a bit. I've got two VMs converged with the following kitchen.yml
, adapted (very slightly) from the cookbook's kitchen.yml:
---
driver:
name: vagrant
network:
- ["forwarded_port", {guest: 9200, host: 9200, auto_correct: true}] # elastic
- ["forwarded_port", {guest: 80, host: 8080, auto_correct: true}] # kibana
- ["forwarded_port", {guest: 443, host: 8443, auto_correct: true}] # ssl
driver_config:
require_chef_omnibus: true
provisioner:
name: chef_zero
data_bags_path: '../../data_bags'
environments_path: '../../environments'
nodes_path: '../../nodes'
roles_path: '../../repo/roles'
client_rb:
environment: development
attributes:
openssh:
server:
password_authentication: 'yes'
permit_root_login: 'yes'
cloud:
public_ipv4: '127.0.0.1'
authorization:
sudo:
users: ['vagrant']
passwordless: true
elasticsearch:
network:
host: '127.0.0.1'
index:
number_of_replicas: 0
# so the kitchen node doesn't have unallocated replica shards
# and comes up green (healthy) instead of yellow
discovery:
zen:
minimum_master_nodes: 1 # since search returns more than one, but they are fake
elkstack:
config:
iptables:
enabled: true
kibana:
password: 'kibana'
platforms:
- name: centos-6.5
run_list:
- recipe[yum]
driver_config:
box: centos_6.5_packer
box_url: https://s3.amazonaws.com/o2o-public/centos_6.5_packer_provisionerless.box
suites:
- name: elkserver
driver:
customize:
memory: 3072
run_list:
- recipe[O2O-elk::single-node]
- name: logtest
driver:
run_list:
- recipe[O2O-elk::agent]
Where my wrapper cookbook recipes are just doing what you suggested above (installing java::default + elkstack::default on the server box, and java::default + elkstack::agent on the agent).
The deploys complete successfully, and I am able to access kibana at https://127.0.0.1:8443
. But it doesn't look like my agent box is correctly configured to send data over to the server.
How can I best configure that for a local environment?
Versions:
OS: CentOS 6.5
Chef Development Kit Version: 0.6.2
chef-client version: 12.3.0
berks version: 3.2.4
kitchen version: 1.4.0
Hi @Joseph-R -- test kitchen converges completely independent machines. Each test case on elkstack is a completely self-contained scenario. Test-kitchen shares no data between your agent and server test cases, so they'd never be sending log lines from the agent to the server. Each test case is completely independent and we'd never send logs from the agent test case to the server test case (they're separate tests, and just the way TK works).
For a local environment, elkstack uses chef search to find servers from agents, or it allows you to set a specific master server. If you have an example of an issue in that scenario, I'd be glad to take a look, but test-kitchen won't be a good scenario. It never has supported multi-server tests that can see/reference each other.
It would help allot if there were an example on how to setup and use this recipe.