search

patschi / parsedmarc-dockerized

Dockerized self-initializing parsedmarc docker stack for lazy people
GNU General Public License v3.0
78 stars 21 forks source link

Error with visualization in Kibana #39

Open RPerjans opened 2 months ago

RPerjans commented 2 months ago

I deployed last week, all went well after some tinkering with the parsedmarc config to get it to use MSGraph instead of IMAP. It imports and sort the emails in the mailbox. All seems to be working, but no dashboards have been created in kibana, i can see the indexes from the DMARC but no visualizations. What could be wrong ?

get error: Error encountered while loading saved dashboard: Could not locate that dashboard (id: 269ba470-2871-11e8-b8b2-15742da3055c)

See screenshot

Tobi7889 commented 2 months ago

Same issue here, i think dashboard template is down :/

Duck93157 commented 1 month ago

Same issue here, new install, Ubuntu 24.04.1 image

binomnom commented 2 weeks ago

Same issue here from a fresh install. I noticed the following in the parsedmarc-init-1 logs:

++ curl -X POST 'http://kibana:5601/api/saved_objects/_import?overwrite=true' -H 'kbn-xsrf: true' --form file=@/etc/parsedmarc/kibana_export.ndjson
+ RES='{"statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred."}'

and these three in the kibana-1 container:

{"tags": ["error", "savedobjects-service"],
"message": "TypeError: Invalid Version: 
                at new SemVer (/usr/share/kibana/node_modules/semver/classes/semver.js:38:13)
                at compare (/usr/share/kibana/node_modules/semver/functions/compare.js:3:32)
                at gt (/usr/share/kibana/node_modules/semver/functions/gt.js:2:29)
                at isPre730Panel (/usr/share/kibana/src/plugins/dashboard/common/saved_dashboard_references.js:20:47)
                at Array.some (<anonymous>)
                at extractReferences (/usr/share/kibana/src/plugins/dashboard/common/saved_dashboard_references.js:73:14)
                at /usr/share/kibana/src/plugins/dashboard/server/saved_objects/dashboard_migrations.js:128:59
                at /usr/share/kibana/node_modules/lodash/lodash.js:5177:46
                at tryTransformDoc (/usr/share/kibana/src/core/server/saved_objects/migrations/core/document_migrator.js:563:22)
                at migrateProp (/usr/share/kibana/src/core/server/saved_objects/migrations/core/document_migrator.js:638:22)
                at applyMigrations (/usr/share/kibana/src/core/server/saved_objects/migrations/core/document_migrator.js:376:20)
                at DocumentMigrator.transformAndValidate [as transformDoc] (/usr/share/kibana/src/core/server/saved_objects/migrations/core/document_migrator.js:323:22)
                at DocumentMigrator.migrate (/usr/share/kibana/src/core/server/saved_objects/migrations/core/document_migrator.js:135:16)
                at KibanaMigrator.migrateDocument (/usr/share/kibana/src/core/server/saved_objects/migrations/kibana/kibana_migrator.js:183:34)
                at /usr/share/kibana/src/core/server/saved_objects/service/lib/repository.js:383:74
                at Array.map (<anonymous>)"
}
{"tags": ["error", "http"],
"message": "TransformSavedObjectDocumentError: Migration function for version 7.11.0 threw an error
                at tryTransformDoc (/usr/share/kibana/src/core/server/saved_objects/migrations/core/document_migrator.js:576:13)
                at migrateProp (/usr/share/kibana/src/core/server/saved_objects/migrations/core/document_migrator.js:638:22)
                at applyMigrations (/usr/share/kibana/src/core/server/saved_objects/migrations/core/document_migrator.js:376:20)
                at DocumentMigrator.transformAndValidate [as transformDoc] (/usr/share/kibana/src/core/server/saved_objects/migrations/core/document_migrator.js:323:22)
                at DocumentMigrator.migrate (/usr/share/kibana/src/core/server/saved_objects/migrations/core/document_migrator.js:135:16)
                at KibanaMigrator.migrateDocument (/usr/share/kibana/src/core/server/saved_objects/migrations/kibana/kibana_migrator.js:183:34)
                at /usr/share/kibana/src/core/server/saved_objects/service/lib/repository.js:383:74
                at Array.map (<anonymous>)
                at SavedObjectsRepository.bulkCreate (/usr/share/kibana/src/core/server/saved_objects/service/lib/repository.js:326:49)
                at SavedObjectsClient.bulkCreate (/usr/share/kibana/src/core/server/saved_objects/service/saved_objects_client.js:168:35)
                at EncryptedSavedObjectsClientWrapper.bulkCreate (/usr/share/kibana/x-pack/plugins/encrypted_saved_objects/server/saved_objects/encrypted_saved_objects_client_wrapper.js:63:93)
                at processTicksAndRejections (node:internal/process/task_queues:95:5)
                at SpacesSavedObjectsClient.bulkCreate (/usr/share/kibana/x-pack/plugins/spaces/server/saved_objects/spaces_saved_objects_client.js:70:12)
                at createSavedObjects (/usr/share/kibana/src/core/server/saved_objects/import/lib/create_saved_objects.js:100:32)
                at importSavedObjectsFromStream (/usr/share/kibana/src/core/server/saved_objects/import/import_saved_objects.js:92:36)
                at /usr/share/kibana/src/core/server/saved_objects/routes/import.js:92:22
Caused by:
TypeError: Invalid Version: 
                at new SemVer (/usr/share/kibana/node_modules/semver/classes/semver.js:38:13)
                at compare (/usr/share/kibana/node_modules/semver/functions/compare.js:3:32)
                at gt (/usr/share/kibana/node_modules/semver/functions/gt.js:2:29)
                at isPre730Panel (/usr/share/kibana/src/plugins/dashboard/common/saved_dashboard_references.js:20:47)
                at Array.some (<anonymous>)
                at extractReferences (/usr/share/kibana/src/plugins/dashboard/common/saved_dashboard_references.js:73:14)
                at /usr/share/kibana/src/plugins/dashboard/server/saved_objects/dashboard_migrations.js:128:59
                at /usr/share/kibana/node_modules/lodash/lodash.js:5177:46
                at tryTransformDoc (/usr/share/kibana/src/core/server/saved_objects/migrations/core/document_migrator.js:563:22)
                at migrateProp (/usr/share/kibana/src/core/server/saved_objects/migrations/core/document_migrator.js:638:22)
                at applyMigrations (/usr/share/kibana/src/core/server/saved_objects/migrations/core/document_migrator.js:376:20)
                at DocumentMigrator.transformAndValidate [as transformDoc] (/usr/share/kibana/src/core/server/saved_objects/migrations/core/document_migrator.js:323:22)
                at DocumentMigrator.migrate (/usr/share/kibana/src/core/server/saved_objects/migrations/core/document_migrator.js:135:16)
                at KibanaMigrator.migrateDocument (/usr/share/kibana/src/core/server/saved_objects/migrations/kibana/kibana_migrator.js:183:34)
                at /usr/share/kibana/src/core/server/saved_objects/service/lib/repository.js:383:74
                at Array.map (<anonymous>) {
  originalError: TypeError: Invalid Version: 
                  at new SemVer (/usr/share/kibana/node_modules/semver/classes/semver.js:38:13)
                  at compare (/usr/share/kibana/node_modules/semver/functions/compare.js:3:32)
                  at gt (/usr/share/kibana/node_modules/semver/functions/gt.js:2:29)
                  at isPre730Panel (/usr/share/kibana/src/plugins/dashboard/common/saved_dashboard_references.js:20:47)
                  at Array.some (<anonymous>)
                  at extractReferences (/usr/share/kibana/src/plugins/dashboard/common/saved_dashboard_references.js:73:14)
                  at /usr/share/kibana/src/plugins/dashboard/server/saved_objects/dashboard_migrations.js:128:59
                  at /usr/share/kibana/node_modules/lodash/lodash.js:5177:46
                  at tryTransformDoc (/usr/share/kibana/src/core/server/saved_objects/migrations/core/document_migrator.js:563:22)
                  at migrateProp (/usr/share/kibana/src/core/server/saved_objects/migrations/core/document_migrator.js:638:22)
                  at applyMigrations (/usr/share/kibana/src/core/server/saved_objects/migrations/core/document_migrator.js:376:20)
                  at DocumentMigrator.transformAndValidate [as transformDoc] (/usr/share/kibana/src/core/server/saved_objects/migrations/core/document_migrator.js:323:22)
                  at DocumentMigrator.migrate (/usr/share/kibana/src/core/server/saved_objects/migrations/core/document_migrator.js:135:16)
                  at KibanaMigrator.migrateDocument (/usr/share/kibana/src/core/server/saved_objects/migrations/kibana/kibana_migrator.js:183:34)
                  at /usr/share/kibana/src/core/server/saved_objects/service/lib/repository.js:383:74
                  at Array.map (<anonymous>),
  version: '7.11.0'"
}
{
"type": "error",
"message": "Error: Internal Server Error
                at HapiResponseAdapter.toInternalError (/usr/share/kibana/src/core/server/http/router/response_adapter.js:48:19)
                at Router.handle (/usr/share/kibana/src/core/server/http/router/router.js:164:34)
                at processTicksAndRejections (node:internal/process/task_queues:95:5)
                at handler (/usr/share/kibana/src/core/server/http/router/router.js:122:50)
                at exports.Manager.execute (/usr/share/kibana/node_modules/@hapi/hapi/lib/toolkit.js:60:28)
                at Object.internals.handler (/usr/share/kibana/node_modules/@hapi/hapi/lib/handler.js:46:20)
                at exports.execute (/usr/share/kibana/node_modules/@hapi/hapi/lib/handler.js:31:20)
                at Request._lifecycle (/usr/share/kibana/node_modules/@hapi/hapi/lib/request.js:371:32)
                at Request._execute (/usr/share/kibana/node_modules/@hapi/hapi/lib/request.js:281:9)"
}

could it be that kibana has no permissions to change the dashboard file?

binomnom commented 2 weeks ago

UPDATE: I managed to fix it. The errors I posted are actually useful: The kibana dashboard file is intended for a newer version of kibana, which is visible in "version" tags throughout the "panelsJSON" object in the file. I changed the docker-compose to request elasticsearch and kibana versions 8.10.4 as requested in the dashboard config, and after a complete rebuild it worked.