search

philhagen / sof-elk

Configuration files for the SOF-ELK VM
GNU General Public License v3.0
1.46k stars 272 forks source link

Kape/Filesystem dashboard error #243

Closed zam89 closed 2 years ago

zam89 commented 2 years ago

Hi, I'm having similar problem with this issue #242, where Filesystem dashboard showing shard failed problem. I'm using VM image Public SOF-ELK v20211006.7z.

I followed Phil instruction at https://github.com/philhagen/sof-elk/issues/242#issuecomment-1027526628_; by running sudo sof-elk_update.sh to update. After that, when I tried to run sof-elk_clear.py -I filesystem, the command is incorrect. Probably the -I is actually -i? So I when to delete the filesystem index; by running command sudo sof-elk_clear.py -i filesystem.

After that, it showing this error: sof-elf_filesystem1 No matching indices found for **filesystem-***. I wasn't clear on the instruction how to reload the data as per Phil instructions in previous issue.

Any help is highly appreciated. Thanks!

zam89 commented 2 years ago

I also tried to run -f & -r arg, seems like nothing happen:

[elk_user@sof-elk ~]$ sudo sof-elk_clear.py -f /logstash/kape/host/20220217123410_MFTECmd_\$MFT_Output.json -r
[sudo] password for elk_user:
No matching documents.  Nothing to delete.
will re-load the following files:
Reload these files? [n]|y: y
philhagen commented 2 years ago

the -i is indeed a lowercase. victim of autocorrect.

the reload functionality is flaky, especially for specific files. The most reliable way to reload the data is to create new copy in the ingest directory. I'll often created e.g. /logstash/kape/load_2/ and place the files there. I do need to get deeper into that script to fix that functionality...

With that, and a force reload (shift-click on the browser reload button), the data should appear fine after it's loaded. One possible exception with KAPE filesystem data in particular is that some records don't have all date fields, so you may see an error on a few of the visualization panels, so be aware of that possibility.

Let me know if that does the trick!

zam89 commented 2 years ago

Hi Phil. It's working flawlessly after I copied those files to new folder as you suggested. sof-elf_filesystem2

btw, just sharing. I'm ingesting JSON output from latest KAPE v1.1.0; using these SOFELK modules below. gkape1

KAPE json output image

philhagen commented 2 years ago

Thanks for closing the loop and I'm glad that took care of it for you!