Dashboard and visualizations not working - error with fields.keyword

[Rajprince2793]

We've been using o365 beat v1.4.0 for sometime and have had no issues with the API connection or pulling data from Azure tenant before. Recently we wanted to try the dashboards and visualizations available with the latest version , and so we upgraded to v 1.5.1.

I'm still able to see the content blobs being pulled and the data written to elastic nodes , but the dashboards and visualizations show errors continuously at each step. First , I got this error

*** Could not locate that index-pattern (id: ceb2f990-f469-11e9-9f4d-5dd1f9c9e483), click here to re-create it

I realized it could be that the index-pattern ID might not have reflected properly , and I edited the saved searches ( o365 alerts | o365 unique users (logins) | o365 client ips | o365 user actions ) to reflect the index-pattern ID , index refname and name as o365beat- ( as this seems to be the custom ID for the index-pattern 0365beat- that was automatically created when o365 beat was run for the first time ). Still not sure why it was not able to automatically reference the right index , but atleast after this step this error was gone.

Now I'm getting this error on each of these visualizations :

For visualiztion = o365 unique users (logins) Could not locate that index-pattern-field (id: user.id.keyword)

For visualiztion = o365 user actions Could not locate that index-pattern-field (id: user.id.keyword)

For visualiztion = o365 client ips Could not locate that index-pattern-field (id: client.ip.keyword)

[Rajprince2793]

After fixing the first error by renaming the index pattern ID , one of the dashboard saved searches started working , which is o365 alerts

But the issue seems to be with the remaining ones that are using user.id.keyword and client.ip.keyword.

Any help on this is appreciated. Thanks in advance.

[aymenwerg]

@Rajprince2793 please how do you make a connection with Azure API can you send me please your config file and all steps to fixe my issues ? please contat me ao@clouseteer.de

[Rajprince2793]

@aymenwerg If you're having trouble with the O365 beat config file , you can reference this link below

https://vizion.ai/forum/topic/o365beat-for-windows/

Make sure a custom application ( for SIEM ) is created on Azure AD by following the steps here

https://docs.microsoft.com/en-us/office/office-365-management-api/get-started-with-office-365-management-apis#register-your-application-in-azure-ad

Below is the key piece of the config that you need to obtain from the app you configured

tenant_domain: ${O365BEAT_TENANT_DOMAIN:your tenant domain here} client_secret: ${O365BEAT_CLIENT_SECRET:your client secret here} client_id: ${O365BEAT_CLIENT_ID:your client id here} # aka application id (GUID) directory_id: ${O365BEAT_DIRECTORY_ID:your directory id here} # aka tenant id (GUID) registry_file_path: ${O365BEAT_REGISTRY_PATH:./o365beat-registry.json}

[Rajprince2793]

@chris-counteractive Must have missed to tag you on the post , sorry about that

[chris-counteractive]

In researching another issue, I realized (a few months late) that filebeat now supports o365 with an official module, as of v7.7.0. It supports a variety of visualizations out of the box, and will surely stay more current with the latest updates to the Elastic Stack, including Kibana.

Given there's an "official solution" to visualization, I'm going to close this issue, please let me know if you have any further questions. Thanks!