search

elastic / kibana

Your window into the Elastic Stack
https://www.elastic.co/products/kibana
Other
19.73k stars 8.14k forks source link

Kibana Login Error even after giving valid credentials #162022

Closed rshiva777 closed 1 year ago

rshiva777 commented 1 year ago

Kibana version: kibana:8.5.1

Elasticsearch version: elasticsearch:8.5.1

Server OS version: VMware Photon OS/Linux 4.19.247-7.ph3

Browser version: Chrome

Browser OS version: Windows

Original install method (e.g. download page, yum, from source, etc.): Helm Chart

Describe the bug:

We have installed the elasticsearch and kibana using helm chart. Post installation able to access kibana with elasticsearch credentials user/password: elastic/***

But, after 3 to 5 days, kibana is thowing wrong credentials although given the correct ones

Steps to reproduce:

  1. Install elasticsearch and kibana using helm chart
  2. access kibana using elastic username and password
  3. Try to access again in few days, kibana throwing "Username or password is incorrect. Please try again."

Expected behavior: Kibana should be accessible even after few days

Screenshots (if relevant):

image

Errors in browser console (if relevant):

Provide logs and/or server output (if relevant):

Any additional context:

  1. I am able to authenticate from elasticsearch using curl request with same credentials
  2. If i do restart kibana pod, i am able to login with same credentials
rshiva777 commented 1 year ago

Adding pod logs:

ratnavarapuv@de1gks1tkg:~$ kubectl logs -f -n efk-system-logging kibana-kibana-59f9d79c59-6hbmx [2023-07-15T05:20:48.543+00:00][INFO ][node] Kibana process configured with roles: [background_tasks, ui] [2023-07-15T05:21:08.623+00:00][INFO ][plugins-service] Plugin "cloudExperiments" is disabled. [2023-07-15T05:21:08.697+00:00][INFO ][plugins-service] Plugin "profiling" is disabled. [2023-07-15T05:21:08.828+00:00][INFO ][http.server.Preboot] http server running at http://0.0.0.0:5601 [2023-07-15T05:21:08.885+00:00][INFO ][plugins-system.preboot] Setting up [1] plugins: [interactiveSetup] [2023-07-15T05:21:08.929+00:00][WARN ][config.deprecation] The default mechanism for Reporting privileges will work differently in future versions, which will affect the behavior of this cluster. Set "xpack.reporting.roles.enabled" to "false" to adopt the future behavior before upgrading. [2023-07-15T05:21:09.361+00:00][INFO ][plugins-system.standard] Setting up [125] plugins: [translations,monitoringCollection,licensing,globalSearch,globalSearchProviders,features,mapsEms,licenseApiGuard,usageCollection,taskManager,telemetryCollectionManager,telemetryCollectionXpack,kibanaUsageCollection,share,screenshotMode,banners,newsfeed,guidedOnboarding,fieldFormats,expressions,dataViews,embeddable,uiActionsEnhanced,charts,esUiShared,customIntegrations,home,searchprofiler,painlessLab,grokdebugger,management,advancedSettings,spaces,security,lists,files,encryptedSavedObjects,cloud,snapshotRestore,screenshotting,telemetry,licenseManagement,eventLog,actions,stackConnectors,console,bfetch,data,watcher,reporting,fileUpload,ingestPipelines,alerting,aiops,unifiedSearch,unifiedFieldList,savedSearch,savedObjects,graph,savedObjectsTagging,savedObjectsManagement,presentationUtil,expressionShape,expressionRevealImage,expressionRepeatImage,expressionMetric,expressionImage,controls,eventAnnotation,dataViewFieldEditor,triggersActionsUi,transform,stackAlerts,ruleRegistry,discover,fleet,indexManagement,remoteClusters,crossClusterReplication,indexLifecycleManagement,cloudSecurityPosture,discoverEnhanced,visualizations,canvas,visTypeXy,visTypeVislib,visTypeVega,visTypeTimeseries,rollup,visTypeTimelion,visTypeTagcloud,visTypeTable,visTypeMetric,visTypeHeatmap,visTypeMarkdown,dashboard,dashboardEnhanced,expressionXY,expressionTagcloud,expressionPartitionVis,visTypePie,expressionMetricVis,expressionLegacyMetricVis,expressionHeatmap,expressionGauge,lens,maps,dataVisualizer,cases,timelines,sessionView,kubernetesSecurity,observability,osquery,ml,synthetics,securitySolution,infra,upgradeAssistant,monitoring,logstash,enterpriseSearch,apm,visTypeGauge,dataViewManagement] [2023-07-15T05:21:09.378+00:00][INFO ][plugins.taskManager] TaskManager is identified by the Kibana UUID: 70a6a6e2-ac44-4ac5-8cdf-b6a1e0e9d9b7 [2023-07-15T05:21:09.511+00:00][WARN ][plugins.security.config] Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command. [2023-07-15T05:21:09.511+00:00][WARN ][plugins.security.config] Session cookies will be transmitted over insecure connections. This is not recommended. [2023-07-15T05:21:09.589+00:00][WARN ][plugins.security.config] Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command. [2023-07-15T05:21:09.590+00:00][WARN ][plugins.security.config] Session cookies will be transmitted over insecure connections. This is not recommended. [2023-07-15T05:21:09.600+00:00][WARN ][plugins.encryptedSavedObjects] Saved objects encryption key is not set. This will severely limit Kibana functionality. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command. [2023-07-15T05:21:09.616+00:00][WARN ][plugins.actions] APIs are disabled because the Encrypted Saved Objects plugin is missing encryption key. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command. [2023-07-15T05:21:09.741+00:00][WARN ][plugins.reporting.config] Generating a random key for xpack.reporting.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.reporting.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command. [2023-07-15T05:21:09.742+00:00][WARN ][plugins.reporting.config] Found 'server.host: "0.0.0.0"' in Kibana configuration. Reporting is not able to use this as the Kibana server hostname. To enable PNG/PDF Reporting to work, 'xpack.reporting.kibanaServer.hostname: localhost' is automatically set in the configuration. You can prevent this message by adding 'xpack.reporting.kibanaServer.hostname: localhost' in kibana.yml. [2023-07-15T05:21:09.786+00:00][WARN ][plugins.alerting] APIs are disabled because the Encrypted Saved Objects plugin is missing encryption key. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command. [2023-07-15T05:21:09.832+00:00][INFO ][plugins.ruleRegistry] Installing common resources shared between all indices [2023-07-15T05:21:09.931+00:00][INFO ][plugins.cloudSecurityPosture] Registered task successfully [Task: cloud_security_posture-stats_task] [2023-07-15T05:21:11.183+00:00][INFO ][plugins.screenshotting.config] Chromium sandbox provides an additional layer of protection, and is supported for Linux Ubuntu 20.04 OS. Automatically enabling Chromium sandbox. [2023-07-15T05:21:12.015+00:00][INFO ][savedobjects-service] Waiting until all Elasticsearch nodes are compatible with Kibana before starting saved objects migrations... [2023-07-15T05:21:12.015+00:00][INFO ][savedobjects-service] Starting saved objects migrations [2023-07-15T05:21:12.107+00:00][INFO ][savedobjects-service] [.kibana_task_manager] INIT -> OUTDATED_DOCUMENTS_SEARCH_OPEN_PIT. took: 12ms. [2023-07-15T05:21:12.111+00:00][INFO ][savedobjects-service] [.kibana] INIT -> OUTDATED_DOCUMENTS_SEARCH_OPEN_PIT. took: 21ms. [2023-07-15T05:21:12.113+00:00][INFO ][savedobjects-service] [.kibana_task_manager] OUTDATED_DOCUMENTS_SEARCH_OPEN_PIT -> OUTDATED_DOCUMENTS_SEARCH_READ. took: 7ms. [2023-07-15T05:21:12.115+00:00][INFO ][savedobjects-service] [.kibana] OUTDATED_DOCUMENTS_SEARCH_OPEN_PIT -> OUTDATED_DOCUMENTS_SEARCH_READ. took: 4ms. [2023-07-15T05:21:12.120+00:00][INFO ][savedobjects-service] [.kibana_task_manager] OUTDATED_DOCUMENTS_SEARCH_READ -> OUTDATED_DOCUMENTS_SEARCH_CLOSE_PIT. took: 7ms. [2023-07-15T05:21:12.123+00:00][INFO ][savedobjects-service] [.kibana] OUTDATED_DOCUMENTS_SEARCH_READ -> OUTDATED_DOCUMENTS_SEARCH_CLOSE_PIT. took: 8ms. [2023-07-15T05:21:12.125+00:00][INFO ][savedobjects-service] [.kibana_task_manager] OUTDATED_DOCUMENTS_SEARCH_CLOSE_PIT -> UPDATE_TARGET_MAPPINGS. took: 5ms. [2023-07-15T05:21:12.127+00:00][INFO ][savedobjects-service] [.kibana] OUTDATED_DOCUMENTS_SEARCH_CLOSE_PIT -> UPDATE_TARGET_MAPPINGS. took: 4ms. [2023-07-15T05:21:12.150+00:00][INFO ][savedobjects-service] [.kibana_task_manager] UPDATE_TARGET_MAPPINGS -> UPDATE_TARGET_MAPPINGS_WAIT_FOR_TASK. took: 25ms. [2023-07-15T05:21:12.187+00:00][INFO ][savedobjects-service] [.kibana] UPDATE_TARGET_MAPPINGS -> UPDATE_TARGET_MAPPINGS_WAIT_FOR_TASK. took: 60ms. [2023-07-15T05:21:12.261+00:00][INFO ][savedobjects-service] [.kibana_task_manager] UPDATE_TARGET_MAPPINGS_WAIT_FOR_TASK -> DONE. took: 111ms. [2023-07-15T05:21:12.262+00:00][INFO ][savedobjects-service] [.kibana_task_manager] Migration completed after 168ms [2023-07-15T05:21:12.593+00:00][INFO ][savedobjects-service] [.kibana] UPDATE_TARGET_MAPPINGS_WAIT_FOR_TASK -> DONE. took: 406ms. [2023-07-15T05:21:12.593+00:00][INFO ][savedobjects-service] [.kibana] Migration completed after 503ms [2023-07-15T05:21:12.598+00:00][INFO ][plugins-system.preboot] Stopping all plugins. [2023-07-15T05:21:12.599+00:00][INFO ][plugins-system.standard] Starting [125] plugins: [translations,monitoringCollection,licensing,globalSearch,globalSearchProviders,features,mapsEms,licenseApiGuard,usageCollection,taskManager,telemetryCollectionManager,telemetryCollectionXpack,kibanaUsageCollection,share,screenshotMode,banners,newsfeed,guidedOnboarding,fieldFormats,expressions,dataViews,embeddable,uiActionsEnhanced,charts,esUiShared,customIntegrations,home,searchprofiler,painlessLab,grokdebugger,management,advancedSettings,spaces,security,lists,files,encryptedSavedObjects,cloud,snapshotRestore,screenshotting,telemetry,licenseManagement,eventLog,actions,stackConnectors,console,bfetch,data,watcher,reporting,fileUpload,ingestPipelines,alerting,aiops,unifiedSearch,unifiedFieldList,savedSearch,savedObjects,graph,savedObjectsTagging,savedObjectsManagement,presentationUtil,expressionShape,expressionRevealImage,expressionRepeatImage,expressionMetric,expressionImage,controls,eventAnnotation,dataViewFieldEditor,triggersActionsUi,transform,stackAlerts,ruleRegistry,discover,fleet,indexManagement,remoteClusters,crossClusterReplication,indexLifecycleManagement,cloudSecurityPosture,discoverEnhanced,visualizations,canvas,visTypeXy,visTypeVislib,visTypeVega,visTypeTimeseries,rollup,visTypeTimelion,visTypeTagcloud,visTypeTable,visTypeMetric,visTypeHeatmap,visTypeMarkdown,dashboard,dashboardEnhanced,expressionXY,expressionTagcloud,expressionPartitionVis,visTypePie,expressionMetricVis,expressionLegacyMetricVis,expressionHeatmap,expressionGauge,lens,maps,dataVisualizer,cases,timelines,sessionView,kubernetesSecurity,observability,osquery,ml,synthetics,securitySolution,infra,upgradeAssistant,monitoring,logstash,enterpriseSearch,apm,visTypeGauge,dataViewManagement] [2023-07-15T05:21:15.512+00:00][INFO ][plugins.monitoring.monitoring] config sourced from: production cluster [2023-07-15T05:21:17.904+00:00][INFO ][http.server.Kibana] http server running at http://0.0.0.0:5601 [2023-07-15T05:21:18.025+00:00][INFO ][plugins.ruleRegistry] Installed common resources shared between all indices [2023-07-15T05:21:18.026+00:00][INFO ][plugins.ruleRegistry] Installing resources for index .alerts-observability.uptime.alerts [2023-07-15T05:21:18.026+00:00][INFO ][plugins.ruleRegistry] Installing resources for index .alerts-security.alerts [2023-07-15T05:21:18.026+00:00][INFO ][plugins.ruleRegistry] Installing resources for index .preview.alerts-security.alerts [2023-07-15T05:21:18.027+00:00][INFO ][plugins.ruleRegistry] Installing resources for index .alerts-observability.logs.alerts [2023-07-15T05:21:18.027+00:00][INFO ][plugins.ruleRegistry] Installing resources for index .alerts-observability.metrics.alerts [2023-07-15T05:21:18.027+00:00][INFO ][plugins.ruleRegistry] Installing resources for index .alerts-observability.apm.alerts [2023-07-15T05:21:18.121+00:00][INFO ][plugins.monitoring.monitoring.kibana-monitoring] Starting monitoring stats collection [2023-07-15T05:21:18.122+00:00][INFO ][plugins.fleet] Beginning fleet setup [2023-07-15T05:21:18.136+00:00][INFO ][status] Kibana is now degraded [2023-07-15T05:21:18.155+00:00][INFO ][plugins.ruleRegistry] Installed resources for index .alerts-observability.apm.alerts [2023-07-15T05:21:18.156+00:00][INFO ][plugins.ruleRegistry] Installed resources for index .alerts-observability.metrics.alerts [2023-07-15T05:21:18.157+00:00][INFO ][plugins.ruleRegistry] Installed resources for index .alerts-observability.logs.alerts [2023-07-15T05:21:18.159+00:00][INFO ][plugins.ruleRegistry] Installed resources for index .alerts-security.alerts [2023-07-15T05:21:18.159+00:00][INFO ][plugins.ruleRegistry] Installed resources for index .alerts-observability.uptime.alerts [2023-07-15T05:21:18.189+00:00][INFO ][plugins.ruleRegistry] Installed resources for index .preview.alerts-security.alerts [2023-07-15T05:21:18.729+00:00][INFO ][plugins.screenshotting.chromium] Browser executable: /usr/share/kibana/x-pack/plugins/screenshotting/chromium/headless_shell-linux_x64/headless_shell [2023-07-15T05:21:18.866+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: scheduled with interval 1h [2023-07-15T05:21:21.406+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-15T05:21:24.008+00:00][INFO ][status] Kibana is now available (was degraded) [2023-07-15T05:25:39.258+00:00][ERROR][plugins.fleet] Failed to fetch latest version of synthetics from registry: Error connecting to package registry: request to https://epr.elastic.co/search?package=synthetics&experimental=true&kibana.version=8.5.1 failed, reason: connect ETIMEDOUT 34.120.127.130:443 [2023-07-15T05:25:39.320+00:00][INFO ][plugins.fleet] Fleet setup completed [2023-07-15T05:25:39.327+00:00][INFO ][plugins.securitySolution] Dependent plugin setup complete - Starting ManifestTask [2023-07-15T05:30:01.401+00:00][ERROR][plugins.fleet] Failed to fetch latest version of synthetics from registry: Error connecting to package registry: request to https://epr.elastic.co/search?package=synthetics&experimental=true&kibana.version=8.5.1 failed, reason: connect ETIMEDOUT 34.120.127.130:443 [2023-07-15T05:30:01.405+00:00][INFO ][plugins.synthetics] Installed synthetics index templates [2023-07-15T05:55:09.318+00:00][INFO ][plugins.securitySolution.endpoint:metadata-check-transforms-task:0.0.1] no endpoint installation found [2023-07-15T06:21:21.491+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-15T07:21:21.840+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-15T07:55:09.917+00:00][INFO ][plugins.securitySolution.endpoint:metadata-check-transforms-task:0.0.1] no endpoint installation found [2023-07-15T08:21:22.498+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-15T09:21:25.262+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-15T09:55:10.386+00:00][INFO ][plugins.securitySolution.endpoint:metadata-check-transforms-task:0.0.1] no endpoint installation found [2023-07-15T10:21:25.571+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-15T11:21:25.882+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-15T11:55:10.936+00:00][INFO ][plugins.securitySolution.endpoint:metadata-check-transforms-task:0.0.1] no endpoint installation found [2023-07-15T12:21:26.072+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-15T13:21:26.260+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-15T13:55:11.375+00:00][INFO ][plugins.securitySolution.endpoint:metadata-check-transforms-task:0.0.1] no endpoint installation found [2023-07-15T14:21:26.575+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-15T15:21:26.963+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-15T15:55:12.076+00:00][INFO ][plugins.securitySolution.endpoint:metadata-check-transforms-task:0.0.1] no endpoint installation found [2023-07-15T16:21:27.186+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-15T17:21:27.464+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-15T17:55:12.535+00:00][INFO ][plugins.securitySolution.endpoint:metadata-check-transforms-task:0.0.1] no endpoint installation found [2023-07-15T18:21:27.679+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-15T19:21:27.879+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-15T19:55:13.071+00:00][INFO ][plugins.securitySolution.endpoint:metadata-check-transforms-task:0.0.1] no endpoint installation found [2023-07-15T20:21:28.143+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-15T21:21:28.335+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-15T21:55:13.417+00:00][INFO ][plugins.securitySolution.endpoint:metadata-check-transforms-task:0.0.1] no endpoint installation found [2023-07-15T22:21:28.539+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-15T23:21:28.799+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-15T23:55:13.844+00:00][INFO ][plugins.securitySolution.endpoint:metadata-check-transforms-task:0.0.1] no endpoint installation found [2023-07-16T00:21:29.080+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-16T01:21:29.316+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-16T01:55:14.505+00:00][INFO ][plugins.securitySolution.endpoint:metadata-check-transforms-task:0.0.1] no endpoint installation found [2023-07-16T02:21:29.621+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-16T03:21:30.035+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-16T03:55:15.044+00:00][INFO ][plugins.securitySolution.endpoint:metadata-check-transforms-task:0.0.1] no endpoint installation found [2023-07-16T04:21:32.080+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-16T05:21:33.354+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-16T05:55:15.456+00:00][INFO ][plugins.securitySolution.endpoint:metadata-check-transforms-task:0.0.1] no endpoint installation found [2023-07-16T06:21:33.627+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-16T07:21:33.936+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-16T07:55:16.037+00:00][INFO ][plugins.securitySolution.endpoint:metadata-check-transforms-task:0.0.1] no endpoint installation found [2023-07-16T08:21:34.124+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-16T09:21:34.345+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-16T09:55:16.463+00:00][INFO ][plugins.securitySolution.endpoint:metadata-check-transforms-task:0.0.1] no endpoint installation found [2023-07-16T10:21:34.553+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-16T11:21:34.764+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-16T11:55:17.088+00:00][INFO ][plugins.securitySolution.endpoint:metadata-check-transforms-task:0.0.1] no endpoint installation found [2023-07-16T12:21:35.307+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-16T13:21:35.336+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-16T13:55:17.579+00:00][INFO ][plugins.securitySolution.endpoint:metadata-check-transforms-task:0.0.1] no endpoint installation found [2023-07-16T14:21:35.633+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-16T15:21:35.884+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-16T15:55:18.097+00:00][INFO ][plugins.securitySolution.endpoint:metadata-check-transforms-task:0.0.1] no endpoint installation found [2023-07-16T16:21:36.115+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-16T17:21:36.481+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-16T17:55:18.637+00:00][INFO ][plugins.securitySolution.endpoint:metadata-check-transforms-task:0.0.1] no endpoint installation found [2023-07-16T18:21:36.782+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-16T19:21:37.087+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-16T19:55:19.220+00:00][INFO ][plugins.securitySolution.endpoint:metadata-check-transforms-task:0.0.1] no endpoint installation found [2023-07-16T20:21:37.427+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-16T21:21:37.774+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-16T21:55:20.038+00:00][INFO ][plugins.securitySolution.endpoint:metadata-check-transforms-task:0.0.1] no endpoint installation found [2023-07-16T22:21:38.108+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-16T23:21:38.367+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-16T23:55:20.579+00:00][INFO ][plugins.securitySolution.endpoint:metadata-check-transforms-task:0.0.1] no endpoint installation found [2023-07-17T00:21:38.652+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-17T01:21:38.902+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-17T01:55:21.092+00:00][INFO ][plugins.securitySolution.endpoint:metadata-check-transforms-task:0.0.1] no endpoint installation found [2023-07-17T02:21:39.129+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-17T03:21:39.405+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-17T03:21:47.399+00:00][INFO ][plugins.security.routes] Logging in with provider "basic" (basic) [2023-07-17T03:22:03.749+00:00][INFO ][plugins.security.routes] Logging in with provider "basic" (basic) [2023-07-17T03:22:41.442+00:00][INFO ][plugins.security.routes] Logging in with provider "basic" (basic) [2023-07-17T03:22:56.323+00:00][INFO ][plugins.security.routes] Logging in with provider "basic" (basic) [2023-07-17T03:23:09.552+00:00][INFO ][plugins.security.routes] Logging in with provider "basic" (basic) [2023-07-17T03:55:21.608+00:00][INFO ][plugins.securitySolution.endpoint:metadata-check-transforms-task:0.0.1] no endpoint installation found [2023-07-17T04:21:39.706+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-17T05:21:40.213+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-17T05:26:09.625+00:00][WARN ][plugins.kibanaUsageCollection.usage-counters-rollup] Failed to rollup Usage Counters saved objects. [2023-07-17T05:26:09.626+00:00][WARN ][plugins.kibanaUsageCollection.usage-counters-rollup] ConnectionError: unable to verify the first certificate at KibanaTransport.request (/usr/share/kibana/node_modules/@elastic/transport/lib/Transport.js:525:31) at runMicrotasks () at processTicksAndRejections (node:internal/process/task_queues:96:5) [2023-07-17T05:55:22.419+00:00][INFO ][plugins.securitySolution.endpoint:metadata-check-transforms-task:0.0.1] no endpoint installation found [2023-07-17T06:14:08.455+00:00][INFO ][plugins.security.routes] Logging in with provider "basic" (basic) [2023-07-17T06:16:38.613+00:00][INFO ][plugins.security.routes] Logging in with provider "basic" (basic) [2023-07-17T06:20:01.754+00:00][INFO ][plugins.security.routes] Logging in with provider "basic" (basic) [2023-07-17T06:21:40.507+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-17T06:22:43.341+00:00][INFO ][plugins.security.routes] Logging in with provider "basic" (basic) [2023-07-17T06:46:20.930+00:00][INFO ][plugins.security.routes] Logging in with provider "basic" (basic) [2023-07-17T06:47:32.160+00:00][INFO ][plugins.security.routes] Logging in with provider "basic" (basic) [2023-07-17T07:21:40.801+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-17T07:24:11.458+00:00][INFO ][plugins.security.routes] Logging in with provider "basic" (basic) [2023-07-17T07:24:22.560+00:00][INFO ][plugins.security.routes] Logging in with provider "basic" (basic) [2023-07-17T07:53:01.242+00:00][INFO ][plugins.security.routes] Logging in with provider "basic" (basic) [2023-07-17T07:54:32.707+00:00][INFO ][plugins.security.routes] Logging in with provider "basic" (basic) [2023-07-17T07:54:44.264+00:00][INFO ][plugins.security.routes] Logging in with provider "basic" (basic) [2023-07-17T07:55:25.398+00:00][INFO ][plugins.securitySolution.endpoint:metadata-check-transforms-task:0.0.1] no endpoint installation found [2023-07-17T07:55:51.908+00:00][INFO ][plugins.security.routes] Logging in with provider "basic" (basic) [2023-07-17T07:57:02.641+00:00][INFO ][plugins.security.routes] Logging in with provider "basic" (basic) [2023-07-17T07:57:30.318+00:00][INFO ][plugins.security.routes] Logging in with provider "basic" (basic) [2023-07-17T07:57:35.748+00:00][INFO ][plugins.security.routes] Logging in with provider "basic" (basic) [2023-07-17T08:21:41.080+00:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization [2023-07-17T08:55:57.939+00:00][INFO ][plugins.security.routes] Logging in with provider "basic" (basic)

[2023-07-17T08:56:15.732+00:00][INFO ][plugins.security.routes] Logging in with provider "basic" (basic)

elasticmachine commented 1 year ago

Pinging @elastic/kibana-security (Team:Security)

azasypkin commented 1 year ago

Hi @rshiva777 ,

The GitHub issues are intended for bug reports, enhancement requests and etc. The Helm charts specific issue you have fits better either for https://github.com/elastic/helm-charts or our Discuss forum, would you mind posting your question there instead? There much more users like you that can help and probably already solved the problem you have.

Closing this issue for now.

I am able to authenticate from elasticsearch using curl request with same credentials

That's weird, Kibana doesn't use credentials, it just forwards them to the Elasticsearch for validation. Does curl to Kibana endpoint work? Does login via private tab works?

rshiva777 commented 1 year ago

Hello @azasypkin ,

This is not the issue with helm chart, it is with kibana application.

Why kibana after a certain period not accepting the elastic search user credentials even correct ones are provided ? I can verify that same credentials are working using curl request to elasticsearch, but when loggging into kibana it is not working.

azasypkin commented 1 year ago

Why kibana after a certain period not accepting the elastic search user credentials even correct ones are provided ?

As I said in https://github.com/elastic/kibana/issues/162022#issuecomment-1638256311, Kibana doesn't validate user credentials, Elasticsearch instance Kibana connected to does that instead.

Does curl to Kibana endpoint work? Does login via private tab work?

Can you answer these questions? If the answer to both questions is No - please enable verbose logging and check if you can notice anything suspicious. But again, the issue you have is more suitable for Discuss forum, it's highly unlikely it's the bug in Kibana.

rshiva777 commented 1 year ago

@azasypkin Yes, correct.

  1. Curl to kibana endpoint workds
root@de1gks1tkg:~$ curl http://kibana.test.com/login -vv
*   Trying 11.99.149.3...
* TCP_NODELAY set
* Connected to kibana.test.com (11.99.149.3) port 80 (#0)
> GET /login HTTP/1.1
> Host: kibana.test.com
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 200 OK
< content-security-policy: script-src 'self' 'unsafe-eval'; worker-src blob: 'self'; style-src 'unsafe-inline' 'self'
< x-content-type-options: nosniff
< referrer-policy: no-referrer-when-downgrade
< kbn-name: kibana-kibana-99694f4c8-jtgrz
< kbn-license-sig: 744c220a2948b77b445ec62ec0f9fbc2df6021e98d3a7bf626499e1d0d962acf
< content-type: text/html; charset=utf-8
< cache-control: private, no-cache, no-store, must-revalidate
< content-length: 91469
< vary: accept-encoding, Accept-Encoding
< accept-ranges: bytes
< date: Tue, 25 Jul 2023 13:16:07 GMT
< x-envoy-upstream-service-time: 12
< server: envoy
  1. Able to authenticate to elasticsearch endpoint directly using curl

i able to authenticate to elasticsearch from the curl directly using the same credentials. When i try to authenticate using kibana it shows the "username or password incorrect" error. If i do restart the kibana pod, then login works. I need to follow this temporary solution when ever login gets fail

Could you please let me know where can i enable the verbose ?

azasypkin commented 1 year ago

Curl to kibana endpoint workds curl http://kibana.test.com/login -vv

Can you try to curl an API Kibana endpoint (or just /, because /login doesn't require credentials) with the credentials, the same way you test with Elasticsearch?

Does login via private tab work?

What about this my question?

Could you please let me know where can i enable the verbose ?

In Kinana config file or wherever you configure Kibana when you install it with Helm Chart(value should be logging.root.level: debug).