[BUG] Missing indices when opening dashboards

[MichaelGibsonAltrad]

Describe the issue Brand new install of LME 0.3 for a user who doesn't have much Linux knowledge using your convenience scripts. Install went fine without any issues or errors and I can access the kibana portal and see data from the test batch of machines which are checking in.

When opening various dashboards, we are seeing "Could not locate that index-pattern-field (id:". The config file imported correctly and I did reimport it successfully but this made no difference.

To Reproduce Steps to reproduce the behavior:

1. Go to Dashboard
2. Click on 'Security Dashboard'
3. Click on 'Threats'
4. Error in section 'Temporary Files in Downloads Folder' -> 'Could not locate that index-pattern-field (id: winlog.event_data.TargetFilename.keyword)'
5. Error in section 'DNS Overview' -> 'Could not locate that index-pattern-field (id: winlog.event_data.QueryName.keyword)'
6. Error in section 'Non Microsoft processes running in as admin' -> 'Could not locate that index-pattern-field (id: winlog.event_data.Image.keyword)'

There are errors on different dashboards with different index names as well.

Expected behavior Data to appear in the dashboard section.

Windows Event Collector (please complete the following information):

• OS: 2019 Fully patched as of 24/9/2020
• WEC Config [e.g. V0.1] : 0.3
• Winlogbeat Config [e.g. V0.1] : 0.3
• Winlogbeat.exe version [e.g. winlogbeat version 6.4.2 (386)] : 7.9.1
• sysmon config [e.g. File hash or Version]: latest via 0.3
• sysmon executable [e.g. V8.1]: 12

Linux Server (please complete the following information):

• Docker: [e.g. Docker version 18.09.3]
• Docker compose stack file version: [e.g. version 0.1]
• Linux: [e.g. PRETTY_NAME="Ubuntu 18.04.2 LTS"] Ubuntu 18.04.5 LTS
• Logstash Version [e.g. #LME logstash config V0.1] 0.3
• Nginx config [e.g. #LME nginx config V0.1] 0.3

Additional context Add any other context about the problem here.

[splurggy]

Yes, will update shortly. Been a bit busy to do much troubleshooting.

On Tue, 13 Apr 2021, 16:27 Duncan, @.***> wrote:

@splurggy https://github.com/splurggy - Are you able to provide anymore information? There isn't much we can really do with the above statement alone.

Thanks,

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ukncsc/lme/issues/77#issuecomment-818826090, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASL6IZWFA3CN4PGRLQZ4JJDTIRPFBANCNFSM4RZH6VYQ .

[adam-ncc]

Closing this issue as the underlying problem should be resolved with the missing indices. For anyone that tested the v0.4 pre-release version it should be safe to move back on to the master branch now as we have merged the relevant changes in.

If there are any future issues with the migration process please feel free open a new issue for us to look at specifically.