[Fleet]: Generic dataset is not displayed for custom logs under Data streams tab

[ghost]

Kibana version: 8.6 Kibana Staging environment

Host OS and Browser version: All, All

Build Details:

Version: 8.6.0 BC5
Build: 58693
Commit: ed40c16ce9999cc47ad55c11bb097d2e443b31a6

Preconditions:

• 8.6 Kibana Staging environment should be available.
• Agent should be enrolled, online with Agent policy.
• Place the file 'rahul_file3.log' in C drive on above windows endpoint where agent is enrolled.

Steps to reproduce:

1. Navigate to Fleet > Agents Policies .
2. Click on Add integration button.
3. Search Custom logs integration.
4. Add Custom logs integration to the Agent policy.
5. Provide the Log file Path(here C:\Program Files\elastic-agent-8.6.0-windows-x86_64\rahul_file3.log) in the Custom logs integration.
6. Navigate to Fleet > Data streams tab.
7. Click on Datasets option and search generic.
8. Observe that 'No options found' is shown on Datasets option on Data streams page.

Expected Result:

• generic dataset should be shown on the Data streams page.

What's working: Custom logs data is shown on the Discover tab > select index logs-*.

Screenshots:

• Data Streams page

• Discover page

[elasticmachine]

Pinging @elastic/fleet (Team:Fleet)

[amolnater-qasource]

Secondary review for this ticket is Done

[ghost]

Hi @kpollich @joshdover,

We have re-validated the issue on the latest 8.7.0 SNAPSHOT and 8.6.1 release build Kibana Staging and Cloud environment respectively and found that the issue is still occurring.

Below are the observations:

• Generic dataset is not displayed for custom logs under Data streams tab.
  • No options found is shown on Datasets dropdown on searching generic on the Data streams page.

Build Details:

Version: 8.7.0 SNAPSHOT
Build: 60186
Commit: 457aeb875025d04b9c1264da1e2c61b7dd95120b

Screenshots:

• 8.7.0 SNAPSHOT:

• 8.6.1:

Please let us know if anything is required from our end.

Thanks!

[amolnater-qasource]

Hi Team,

We have revalidated this issue on latest 8.8 BC3 kibana cloud environment and found it fixed now.

Observations:

• Generic dataset is displayed for custom logs under Data streams tab.

Screenshot:

Build details:

VERSION: 8.8 BC3 kibana cloud environment
BUILD: 62994
COMMIT: 85b22d307ab93fca95c1698ede4cb61d85f3d314

Hence we are closing this issue and marking as QA:Validated.

Thanks!

[ablnk]

I'm reopening the issue because it is reproducible on Serverless environment.