search

opendistro-for-elasticsearch / kibana-reports

Kibana Reports
https://opendistro.github.io/
39 stars 31 forks source link

Error in CSV report #327

Closed Artain closed 3 years ago

Artain commented 3 years ago

Version

Opendistro Elasticsearch 1.13.0 standalone linux Opendistro Kibana 1.13.0 standalone linux

Error description

If the saved discover table includes a null value the csv export will fail with the error message:

[08:16:09.781] [error][opendistroReportsKibana][plugins] Failed to generate report: TypeError: Cannot read property 'toString' of null
  log   [08:16:09.782] [error][opendistroReportsKibana][plugins] TypeError: Cannot read property 'toString' of null
    at sanitize (/srv/opendistroforelasticsearch-kibana/plugins/opendistroReportsKibana/server/routes/utils/dataReportHelpers.ts:227:18)
    at getEsData (/srv/opendistroforelasticsearch-kibana/plugins/opendistroReportsKibana/server/routes/utils/dataReportHelpers.ts:172:34)
    at convertEsDataToCsv (/srv/opendistroforelasticsearch-kibana/plugins/opendistroReportsKibana/server/routes/utils/savedSearchReportHelper.ts:266:18)
    at generateReportData (/srv/opendistroforelasticsearch-kibana/plugins/opendistroReportsKibana/server/routes/utils/savedSearchReportHelper.ts:151:10)
    at process._tickCallback (internal/process/next_tick.js:68:7)
 error  [08:16:07.104]  Error: Internal Server Error
    at HapiResponseAdapter.toError (/srv/opendistroforelasticsearch-kibana/src/core/server/http/router/response_adapter.js:132:19)
    at HapiResponseAdapter.toHapiResponse (/srv/opendistroforelasticsearch-kibana/src/core/server/http/router/response_adapter.js:86:19)
    at HapiResponseAdapter.handle (/srv/opendistroforelasticsearch-kibana/src/core/server/http/router/response_adapter.js:81:17)
    at Router.handle (/srv/opendistroforelasticsearch-kibana/src/core/server/http/router/router.js:164:34)
    at process._tickCallback (internal/process/next_tick.js:68:7)
hollden commented 3 years ago

I have almost the same error on 1.12:

{"type":"log","@timestamp":"2021-02-18T07:56:03Z","tags":["error","plugins","opendistroReportsKibana"],"pid":21258,"message":"Failed to generate report by id: TypeError: Cannot read property 'settings' of undefined"}
{"type":"log","@timestamp":"2021-02-18T07:56:03Z","tags":["error","plugins","opendistroReportsKibana"],"pid":21258,"message":"TypeError: Cannot read property 'settings' of undefined\n    at getMaxResultSize (/usr/share/kibana/plugins/opendistroReportsKibana/server/routes/utils/savedSearchReportHelper.ts:163:35)\n    at process._tickCallback (internal/process/next_tick.js:68:7)"}
Artain commented 3 years ago

@hollden please check https://github.com/opendistro-for-elasticsearch/kibana-reports/pull/313 your error should be fixed with version 1.13.

alexz00 commented 3 years ago

Hi, I'm facing errors too when generating a CSV report:

If I click on "Create report definition" or "Generate CSV" from the Discover Panel, nothing happens and the following exception is traced:

Uncaught TypeError: Invalid attempt to destructure non-iterable instance.
In order to be iterable, non-array objects must have a [Symbol.iterator]() method.
    at _nonIterableRest (:5601/36136/bundles/plugin/opendistroReportsKibana/opendistroReportsKibana.plugin.js:1)
    at _slicedToArray (:5601/36136/bundles/plugin/opendistroReportsKibana/opendistroReportsKibana.plugin.js:1)
    at getTimeFieldsFromUrl (:5601/36136/bundles/plugin/opendistroReportsKibana/opendistroReportsKibana.plugin.js:1)
    at contextMenuCreateReportDefinition (:5601/36136/bundles/plugin/opendistroReportsKibana/opendistroReportsKibana.plugin.js:1)
    at HTMLButtonElement.<anonymous> (:5601/36136/bundles/plugin/opendistroReportsKibana/opendistroReportsKibana.plugin.js:1)
    at HTMLDocument.dispatch (:5601/36136/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:404)
    at HTMLDocument.m.handle (:5601/36136/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:404)

I'm using the ODFE 1.13.0.0 plugin (manually installed on Kibana 7.10.2 OSS).

pySilver commented 3 years ago

I'm getting a little bit different error:

POST https://<redacted>/api/reporting/generateReport?timezone=Europe/Warsaw {"statusCode":500,"error":"Internal Server Error","message":"Cannot read property 'settings' of undefined"}

zhongnansu commented 3 years ago

@pySilver @hollden from the error log this should be related to https://github.com/opendistro-for-elasticsearch/kibana-reports/issues/249, and the fix is included in 1.13.0 release. Please check

pySilver commented 3 years ago

@zhongnansu I've just switched to 1.13.0 (from 1.12.0), reports aren't generating but the error is different now :(

While clicking "Download CSV" no network communication happens and UI overlay does not appear. However browser console now has the following error:

Uncaught TypeError: Cannot read property 'diff' of undefined
    at getTimeFieldsFromUrl (opendistroReportsKibana.plugin.js:1)
    at HTMLButtonElement.<anonymous> (opendistroReportsKibana.plugin.js:1)
    at HTMLDocument.dispatch (kbn-ui-shared-deps.js:404)
    at HTMLDocument.m.handle (kbn-ui-shared-deps.js:404)
zhongnansu commented 3 years ago

@pySilver can you provide the url at the time when you click the "Download csv" button?

zhongnansu commented 3 years ago

@Artain we'll look into this

pySilver commented 3 years ago

@zhongnansu sure. Btw, now for some reason download csv isn't even available (grayed): Screenshot: https://take.ms/M1bh9

URL: 

<kibana.domain>/app/discover#/?notFound=search&notFoundMessage=Could%20not%20locate%20that%20search%20(id:%203540a600-3751-11eb-9c40-cfb624fcd801)&_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now%2Fd,to:now%2Fd))&_a=(columns:!(RequestPath,OriginStatus),filters:!(('$state':(store:appState),meta:(alias:!n,disabled:!f,index:bc899790-374f-11eb-9c40-cfb624fcd801,key:OriginStatus,negate:!f,params:(query:'404'),type:phrase),query:(match_phrase:(OriginStatus:'404')))),index:bc899790-374f-11eb-9c40-cfb624fcd801,interval:auto,query:(language:kuery,query:'request_User-Agent:%20%22Googlebot%22%20and%20(RequestHost:%20%redacted.com%22)%20and%20RequestPath:%20%22%2Fpl%2F%22'),sort:!())
perkons commented 3 years ago

It is still an issue in version 1.13.0 CentOS 8 I am getting this error when trying to Generate CSV:

{"type":"log","@timestamp":"2021-03-03T11:06:29Z","tags":["error","plugins","opendistroReportsKibana"],"pid":41895,"message":"Failed to generate report: TypeError: JSON.parse(...).query.query.replace is not a function"}
{"type":"log","@timestamp":"2021-03-03T11:06:29Z","tags":["error","plugins","opendistroReportsKibana"],"pid":41895,"message":"TypeError: JSON.parse(...).query.query.replace is not a function
at buildQuery (/usr/share/kibana/plugins/opendistroReportsKibana/server/routes/utils/dataReportHelpers.ts:115:18)\n    at getEsDataSize (/usr/share/kibana/plugins/opendistroReportsKibana/server/routes/utils/savedSearchReportHelper.ts:179:22)
at generateReportData (/usr/share/kibana/plugins/opendistroReportsKibana/server/routes/utils/savedSearchReportHelper.ts:138:25)\n    at process._tickCallback (internal/process/next_tick.js:68:7)"}
{"type":"error","@timestamp":"2021-03-03T11:06:27Z","tags":[],"pid":41895,"level":"error","error":{"message":"Internal Server Error","name":"Error","stack":"Error: Internal Server Error
at HapiResponseAdapter.toError (/usr/share/kibana/src/core/server/http/router/response_adapter.js:132:19)\n    at HapiResponseAdapter.toHapiResponse (/usr/share/kibana/src/core/server/http/router/response_adapter.js:86:19)
at HapiResponseAdapter.handle (/usr/share/kibana/src/core/server/http/router/response_adapter.js:81:17)\n    at Router.handle (/usr/share/kibana/src/core/server/http/router/router.js:164:34)
at process._tickCallback (internal/process/next_tick.js:68:7)"},"url":{"protocol":null,"slashes":null,"auth":null,"host":null,"port":null,"hostname":null,"hash":null,"search":"?timezone=Europe/Berlin","query":{"timezone":"Europe/Berlin"},
"pathname":"/api/reporting/generateReport","path":"/api/reporting/generateReport?timezone=Europe/Berlin","href":"/api/reporting/generateReport?timezone=Europe/Berlin"},"message":"Internal Server Error"}

I also did install the dependencies mentioned here https://github.com/opendistro-for-elasticsearch/kibana-reports/blob/dev/kibana-reports/rendering-engine/headless-chrome/README.md#additional-libaries and tested with the headles-chromium, same result.

melvinkokxw commented 3 years ago

This is still an issue in 1.13.0, and I located a possible location for the issue:

https://github.com/opendistro-for-elasticsearch/kibana-reports/blob/0160a02e3a5d26b9ff279deb5831ceb71b5f8893/kibana-reports/server/routes/utils/dataReportHelpers.ts#L224

The sanitize function uses the toString method on null values.

A workaround working for me is to continue if the value is null:

if (doc[field] == null)
    continue
jasonrojas commented 3 years ago

I am also seeing the same thing with the 1.13.2 update.

{"type":"log","@timestamp":"2021-04-16T23:17:26Z","tags":["error","plugins","opendistroReportsKibana"],"pid":30607,"message":"TypeError: JSON.parse(...).query.query.replace is not a function\n at buildQuery (/usr/share/kibana/plugins/opendistroReportsKibana/server/routes/utils/dataReportHelpers.ts:115:18)\n at getEsDataSize (/usr/share/kibana/plugins/opendistroReportsKibana/server/routes/utils/savedSearchReportHelper.ts:179:22)\n at generateReportData (/usr/share/kibana/plugins/opendistroReportsKibana/server/routes/utils/savedSearchReportHelper.ts:138:25)\n at process._tickCallback (internal/process/next_tick.js:68:7)"} {"type":"error","@timestamp":"2021-04-16T23:17:26Z","tags":[],"pid":30607,"level":"error","error":{"message":"Internal Server Error","name":"Error","stack":"Error: Internal Server Error\n at HapiResponseAdapter.toError (/usr/share/kibana/src/core/server/http/router/response_adapter.js:132:19)\n at HapiResponseAdapter.toHapiResponse (/usr/share/kibana/src/core/server/http/router/response_adapter.js:86:19)\n at HapiResponseAdapter.handle (/usr/share/kibana/src/core/server/http/router/response_adapter.js:81:17)\n at Router.handle (/usr/share/kibana/src/core/server/http/router/router.js:164:34)\n at process._tickCallback (internal/process/next_tick.js:68:7)"},"url":{"protocol":null,"slashes":null,"auth":null,"host":null,"port":null,"hostname":null,"hash":null,"search":"?timezone=America/Los_Angeles","query":{"timezone":"America/Los_Angeles"},"pathname":"/api/reporting/generateReport","path":"/api/reporting/generateReport?timezone=America/Los_Angeles","href":"/api/reporting/generateReport?timezone=America/Los_Angeles"},"message":"Internal Server Error"}

Marc-Whiteman commented 3 years ago

I have the same "Internal Server Error" as jasonrojas with ver 1.12.0

zhongnansu commented 3 years ago

This issue has been moved to Dashboards reports. We'll merge in the PR to Opendistro still, and apply the fix to the new repo. For tracking and further discussion, please go to issue in Dashboards-reports https://github.com/opensearch-project/dashboards-reports/issues/40