many Kibana errors due to "message": "Unexpected token < in JSON at position 0", "code": "STREAM" [ solved ]

[commandline-be]

after a simple 'yum upgrade' Kibana is now broken and shows Error on many places on the Welcome [ ROCK ] dashboard

Elasticsearch, Kibana, Filebeat are not at version 7.11 and cannot be downgraded

All errors show the same message [esaggs] > Unexpected token < in JSON at position 0

When clicking on any panel with an Error message and clicking on 'Inspect' the 'View: Requests' field shows a Request starting with { "aggs": { .... } }

for which the response is always

{ "message": "Unexpected token < in JSON at position 0", "code": "STREAM" }

[commandline-be]

root cause: bfetch.plugin.js:1 XHR failed loading: POST "https://donion/internal/bsearch"

[commandline-be]

solved by downgrading Kibana to 7.10, deleting .kibana_3 .kibana_2 and stop/start kibana then i guess i did something wrong or missed something to do, when asked to also delete .kibana_1 all dashboards were gone.

The dasboards were restored using the installation procedure described in the rock-dashboards readme file as in issue https://github.com/rocknsm/rock-dashboards/issues/69

[overscr]

I too received the same "Unexpected token < in JSON at position 0" error after upgrading via yum from 7.9 to 7.12 but it seemed to be a lighttpd issue. I'm not sure deleting the .kibana* indices is a requirement, since going through that process did not resolve the issue. Even after stopping kibana, deleting all .kibana indices, restarting and then restoring all of the ROCK dashboards, the JSON error persisted.

This error was in /var/log/lighttpd/error.log:

(http-header-glue.c.1032) proxy backend sent invalid response header (Transfer-Encoding) to HTTP/1.0 request After switching out lighttpd with nginx, the JSON error went away. To preserve all the original dashboards, visualizations, saved queries, etc, [export-saved-items.sh](https://github.com/rocknsm/rock-dashboards/blob/master/ecs-configuration/kibana/export-saved-items.sh) worked as designed. All the customizations were then restored with the import scripts.

[commandline-be]

thanks for the update, switching off parts is not how i want to progress on this. Sadly this project has gone very dormant. I'm looking how and if so how to continue with RockNSM. It's been more problem than results thus far.