[Fleet]: Logs and metrics are not collected under Datastreams tab on changing inputs.d in inputs.yml for standalone agent.

[harshitgupta-qasource]

Kibana Build details:

VERSION: 8.15.0
BUILD: 75158
COMMIT: 4182d87d4a2edfed7762b48706b6ae885fa6896f

Host OS: All

Preconditions:

1. 8.15.0-SNAPSHOT Kibana cloud environment should be available.
2. Agent Policy should be created.

Steps to reproduce:

1. Navigate to Fleet>Agents tab.
2. Navigate to Agent Policies tab -> Select agent policy ->System integration -> "Config" tab.
3. Copy config provided into standalone agent's inputs.d directory as inputs.yml.
4. Install standalone agent
5. Observe that logs and metrics are not collected under Datastream Tab.

Expected Result: Logs and metrics should get collected under Datastreams tab on changing inputs.d in inputs.yml for standalone agent.

Screen Shot:

• Agent

• Datastream Tab

Feature:

https://github.com/elastic/kibana/issues/180307

[harshitgupta-qasource]

@amolnater-qasource Kindly review

[amolnater-qasource]

Secondary Review for this ticket is Done.

[amolnater-qasource]

FYI @kpollich

[kpollich]

@harshitgupta-qasource Is there anything else in the agent's config, e.g. in its elastic-agent.yml?

Maybe a better set of instructions would be something like

1. Create an agent policy
2. Select "Add agent" and then "Run standalone"
3. Copy the displayed policy to the agent's elastic-agent.yml file
4. Install another integration (e.g. windows on a Windows agent)
5. Navigate to that integration's "Configs" tab
6. Copy the resulting inputs to the agent's inputs.d/input.yml file

Though now I'm realizing we probably also need the output_permissions to be updated for the agent, or it needs to have broader permissions by default. Maybe @flash1293 can help with relaying an example onboarding workflow for standalone agent that we could repurpose to inform our manual test case here?

[flash1293]

@kpollich On any observability serverless project, the /app/observabilityOnboarding/systemLogs/?category=logs should give you a good example :)

[harshitgupta-qasource]

Hi @kpollich

Thanks for providing the step.

We have re-validated this issue on 8.15.0 kibana snapshot cloud environment using following these step:-

Build details: VERSION: 8.15.0 BUILD: 75556 COMMIT: 1e1e35ba08c1066c17dbd482136cbbf63b22c116

Steps to Reproduce

1. Create an agent Policy
2. Click on Add agent and then Select agent Policy
3. after that click on Run Standalone Check mark and Download elastic-agent.yml
4. Copy the displayed policy to the agent's elastic-agent.yml file.
5. Install windows integration to Agent Policy.
6. Navigate to that windows integration's "Configs" tab and Copy configuration.yml into your elastic-agent.yml within inputs.d yml.
7. Run Installation command
8. Navigate to Data-stream Tab (Observe for logs and metricsfor system, elastic-agent and Windows Integration)
9. Then Uninstall the standalone agent.
10. Navigates to Integration tab and Search for nginx integration.
11. Navigate to that nginx integration's "Configs" tab and Copy configuration.yml into your elastic-agent.yml within inputs.d yml
12. Navigate to Data-stream Tab (Observe for logs and metrics for system, elastic-agent, Windows and windows Integration)

Screenshot

• System and Elastic Agent

• Windows and Nginx

We observed that system and elastic-agent metrics are shown under data-stream tab according to their namespace but Windows and nginx integration are displayed with default namespace

Could you please confirm if this is expected?

We have attached the elastic-agent.yml below.

Elastic-agent.yml elastic-agent.zip

[kpollich]

I assume these integrations are all running on the same agent policy with the same namespace settings applied? I'm not sure what would be causing the difference in namespace unless the system and elastic_agent integrations are configured with a custom namespace that deviates from the agent policy setting.

[harshitgupta-qasource]

Hi @kpollich

While creating the agent Policy we have changed the namespace name from default to std_hars.

Then on copying Windows/nginx integration configuration into elastic-agent.yml we observed that the data for newly added config is available with default namespace.

However the data for system integration is available with std_hars.

Please let us know if anything else is required from our end.

Thanks