Autodiscover is not available in any tag release

[basgeerinck]

#291 is available in master, but not in any release (4.9.0. 4.10 or. 4.11.0.)

[pcfens]

I don't use the autodiscover feature, so I can't independently verify it works, but I do see where there are autodiscover parameters in at least 4.11.0.

Can you check to see it's not showing up on your install? If it's there but not working then maybe we need to correct the documentation?

[basgeerinck]

The current (4.11.0) creates an autoconfigure section in the root of the config file, this is working. Filebeat completely ignores the section however (I lost a lot of time here).

I manualy moved the section to the filebeat part of the filebeat.yml, suddenly stuff started responding. Just one message in the logs, the rest is about harvesters:

Oct 20 14:24:11 p-issc-014613 filebeat[818161]: 2022-10-20T14:24:11.042+0200#011INFO#011[autodiscover]#011autodiscover/autodiscover.go:117#011Starting autodiscover manager

As per #291 the filebeat documentation still states:

You define autodiscover settings in the filebeat.autodiscover section of the filebeat.yml config file. To enable autodiscover, you specify a list of providers.

see 7.17 docmentation. Also in v8

With release 4.11.0 of the module I get:

Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Retrieving locales
Info: Loading facts
Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Resource Statement, Class[Filebeat]: has no parameter named 'filebeat.autodiscover' (file: /etc/puppetlabs/code/environments/production/lucimodules/lu_module_issc/manifests/cla_client.pp, line: 197, column: 5) on node p-issc-014613.infra.leidenuniv.nl
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run

I'm using filebeat 7.16.1 and wil be upgrading to 7.17.6.

[basgeerinck]

do you need more information?

[pcfens]

It sounds like we need to move the autodiscover section in the hash that we render into a config file. Is that right?

If that's right I can try to fix it next week (I'm at kubecon) or I'm happy accept a PR if you or anyone else here has time.

[basgeerinck]

291 looks promising.

In 4.9.0 autodiscover was introduced but at the wrong section. https://github.com/pcfens/puppet-filebeat/blob/v4.9.0/manifests/config.pp

Up to 4.11.0 it is still in the wrong section ( Jun 7, 2021 ) https://github.com/pcfens/puppet-filebeat/blob/v4.11.0/manifests/config.pp

At this moment master seems to contain the fix #291: ([image: @pcfens] https://github.com/pcfens pcfens https://github.com/pcfens merged commit 2eb5d27 https://github.com/pcfens/puppet-filebeat/commit/2eb5d270793d659870307b49798bb20cd2070237 into pcfens:master on Nov 19, 2021 https://github.com/pcfens/puppet-filebeat/pull/291#event-5646934380 ) https://github.com/pcfens/puppet-filebeat/blob/master/manifests/config.pp

v4.11.0 released on Jun 7, 2021 (this is before Nov 19 2021)

It seems to me, you just have to create a new tag release.

Regards, Bas

Op wo 26 okt. 2022 om 23:53 schreef Phil Fenstermacher < @.***>:

It sounds like we need to move the autodiscover section in the hash that we render into a config file. Is that right?

If that's right I can try to fix it next week (I'm at kubecon) or I'm happy accept a PR if you or anyone else here has time.

— Reply to this email directly, view it on GitHub https://github.com/pcfens/puppet-filebeat/issues/308#issuecomment-1292696071, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEWSUGLDB7I54DGYEJZTLIDWFGR5FANCNFSM6AAAAAARKEROHU . You are receiving this because you authored the thread.Message ID: @.***>

[pcfens]

I just released v4.12.0 to the forge.

[basgeerinck]

Seems to work > autodiscover is moved to filebeat section.

Notice: /Stage[main]/Filebeat::Config/File[filebeat.yml]/content:
--- /etc/filebeat/filebeat.yml  2022-11-09 14:01:52.800403177 +0100
+++ /tmp/puppet-file20221111-252559-1ie505y     2022-11-11 08:26:47.966686511 +0100
@@ -21,7 +21,6 @@
   config.modules:
     enabled: true
     path: "/etc/filebeat/modules.d/*.yml"
-  shutdown_timeout: '0'
   modules:
   - module: system
     syslog:
@@ -30,10 +29,13 @@
       - "/var/log/messages"
       - "/var/log/secure"
       - "/var/log/cron"
+  overwrite_pipelines: false
+  shutdown_timeout: '0'
   registry:
     path: "/var/lib/filebeat"
     file_permissions: '0600'
     flush: 0s
+  autodiscover: {}
 http: {}
 cloud: {}
 output:
@@ -48,7 +50,6 @@
     loadbalance: true
 shipper: {}
 logging: {}
-autodiscover: {}
 runoptions: {}
 processors:
 - add_host_metadata: {}

Thank you!