Fix libbeat argument parsing to allow beats to be imported in other code bases.

[leehinman]

• https://github.com/elastic/beats/pull/41151
• https://github.com/elastic/beats/pull/41152

had to be reverted because of APM issues:

• https://github.com/elastic/apm-server/pull/14214
• https://github.com/elastic/apm-server/issues/14280

Also when trying to add beats to elastic-agent we discovered:

• https://github.com/elastic/beats/pull/41136

We need to rework Beats argument parsing and init calls to make libbeat both safe to import in other code bases but also provide guidance for APM to use the modified code correctly.

Locations that need updating:

libbeat

• https://github.com/elastic/beats/blob/6d4fbfc3487508bb5af764b428a46259fea10909/libbeat/plugin/cli.go#L55
• https://github.com/elastic/beats/blob/2ff38df906203ba041d1f5492a6007809ad0b066/libbeat/cfgfile/cfgfile.go#L54-L64
• https://github.com/elastic/beats/blob/2ff38df906203ba041d1f5492a6007809ad0b066/libbeat/common/fleetmode/fleet_mode.go#L29-L49
• https://github.com/elastic/beats/blob/2ff38df906203ba041d1f5492a6007809ad0b066/libbeat/statestore/internal/storecompliance/storecompliance.go#L48-L51
• https://github.com/elastic/beats/blob/2ff38df906203ba041d1f5492a6007809ad0b066/libbeat/cmd/root.go#L36-L43
• https://github.com/elastic/beats/blob/2ff38df906203ba041d1f5492a6007809ad0b066/libbeat/publisher/pipeline/module.go#L53-L55
• https://github.com/elastic/beats/blob/2ff38df906203ba041d1f5492a6007809ad0b066/libbeat/cmd/root.go#L98-L115
• https://github.com/elastic/beats/blob/2ff38df906203ba041d1f5492a6007809ad0b066/libbeat/cmd/instance/beat.go#L979-L982
• https://github.com/elastic/beats/blob/2ff38df906203ba041d1f5492a6007809ad0b066/libbeat/cmd/run.go#L44-L47
• https://github.com/elastic/beats/blob/2ff38df906203ba041d1f5492a6007809ad0b066/libbeat/common/config.go#L35

filebeat

• https://github.com/elastic/beats/blob/6339a4292da1bfcf5a33c8593b41556efbbdad86/filebeat/fileset/flags.go#L28-L32
• https://github.com/elastic/beats/blob/6339a4292da1bfcf5a33c8593b41556efbbdad86/filebeat/cmd/root.go#L50-L52
• https://github.com/elastic/beats/blob/6339a4292da1bfcf5a33c8593b41556efbbdad86/filebeat/cmd/root.go#L66-L74
• https://github.com/elastic/beats/blob/6339a4292da1bfcf5a33c8593b41556efbbdad86/filebeat/beater/filebeat.go#L67

metricbeat

• https://github.com/elastic/beats/blob/6339a4292da1bfcf5a33c8593b41556efbbdad86/metricbeat/internal/sysinit/init.go#L31
• https://github.com/elastic/beats/blob/6339a4292da1bfcf5a33c8593b41556efbbdad86/metricbeat/cmd/root.go#L62-L63
• https://github.com/elastic/beats/blob/6339a4292da1bfcf5a33c8593b41556efbbdad86/metricbeat/helper/windows/run.go#L35-L42
• https://github.com/elastic/beats/blob/6339a4292da1bfcf5a33c8593b41556efbbdad86/metricbeat/helper/windows/pdh/mkpdh_defs.go#L33-L35

packetbeat

• https://github.com/elastic/beats/blob/6339a4292da1bfcf5a33c8593b41556efbbdad86/packetbeat/cmd/root.go#L51-L57
• https://github.com/elastic/beats/blob/6339a4292da1bfcf5a33c8593b41556efbbdad86/packetbeat/beater/packetbeat.go#L61-L67

x-pack

• https://github.com/elastic/beats/blob/6339a4292da1bfcf5a33c8593b41556efbbdad86/x-pack/filebeat/module/cisco/shared/gen-ftd-ecs-mapping.go#L22-L23
• https://github.com/elastic/beats/blob/6339a4292da1bfcf5a33c8593b41556efbbdad86/x-pack/metricbeat/module/panw/client.go#L36-L40
• https://github.com/elastic/beats/blob/6339a4292da1bfcf5a33c8593b41556efbbdad86/x-pack/filebeat/module/cisco/shared/gen-ftd-ecs-mapping.go#L22-L23
• https://github.com/elastic/beats/blob/6339a4292da1bfcf5a33c8593b41556efbbdad86/x-pack/filebeat/input/netflow/fields_gen.go#L26-L29
• https://github.com/elastic/beats/blob/6339a4292da1bfcf5a33c8593b41556efbbdad86/x-pack/filebeat/input/netflow/decoder/fields/gen.go#L20-L27

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)