search

elastic / kibana

Your window into the Elastic Stack
https://www.elastic.co/products/kibana
Other
19.82k stars 8.2k forks source link

Data path not honored for Chromium sandbox #140394

Open peterpramb opened 2 years ago

peterpramb commented 2 years ago

Kibana version: 7.17.6

Server OS version: RHEL 7

Original install method (e.g. download page, yum, from source, etc.): Archive

Describe the bug: In our setup the distribution directory is not writable by the user running Kibana. Instead Kibana has been configured with a data path writable for the runtime user:

Unfortunately at startup the configured data path is ignored for the Chromium sandbox (and only for it), causing the following log error:

Ironically, when additionally defining the deprecated environment variable DATA_PATH="/var/lib/kibana" at startup, it works as expected:

$ ls /var/lib/kibana  
chromium-GdJEGP  uuid
peterpramb commented 2 years ago

Unfortunately the workaround using DATA_PATH interferes with the Kibana keystore, which is now searched in DATA_PATH instead of KBN_PATH_CONF...

elasticmachine commented 2 years ago

Pinging @elastic/kibana-operations (Team:Operations)

jbudz commented 2 years ago

Thanks for the detailed writeup @peterpramb - taking a look.

elasticmachine commented 2 years ago

Pinging @elastic/kibana-app-services (Team:AppServicesUx)

peterpramb commented 2 years ago

There is a better workaround than using DATA_PATH which will not impact the keystore:

--- node_modules/@kbn/utils/target_node/path/index.js.orig      2022-08-22 13:17:51.000000000 +0200
+++ node_modules/@kbn/utils/target_node/path/index.js   2022-09-12 10:06:23.696696286 +0200
@@ -28,7 +28,7 @@ process.env.CONFIG_PATH, // deprecated
const CONFIG_DIRECTORIES = [process.env.KBN_PATH_CONF, process.env.KIBANA_PATH_CONF, // deprecated
(0, _path.join)(_repo_root.REPO_ROOT, 'config'), '/etc/kibana'].filter(isString);
const DATA_PATHS = [process.env.DATA_PATH, // deprecated
-(0, _path.join)(_repo_root.REPO_ROOT, 'data'), '/var/lib/kibana'].filter(isString);
+'/var/lib/kibana'].filter(isString);

function findFile(paths) {
   const availablePath = paths.find(configPath => {
elasticmachine commented 9 months ago

Pinging @elastic/appex-sharedux (Team:SharedUX)

tsullivan commented 9 months ago

@peterpramb can we get an idea of how critically (or not) this is negatively impacting your configuration of reporting?

peterpramb commented 9 months ago

I don't remember anymore if this impacted reporting as a whole, but at least the sandbox couldn't be enabled without a writable distribution directory.

vadimkibana commented 9 months ago

Somebody from SharedUx will take a deeper look, will try to reproduce on Linux distro.

Ikuni17 commented 9 months ago

To add a bit of context here, we have merged this fix twice and reverted it twice due to issues it creates. See #158426 for more details.

tsullivan commented 9 months ago

I might not understand the issue fully here, so let me try to recap and please correct me.

  1. kibana.yml has a setting called path.data. In cases when the distribution directory is not writable by the user running the Kibana process, this setting must be customized to a path that is writable by the user. In this case, it's customized to /var/lib/kibana
  2. The screenshotting plugin must be aware of the configured path.data as the location to store the temporary files needed to store the user profile.
  3. Screenshotting gets the root data path using getDataPath, imported from @kbn/utils. It sets the path for the user profile by appending /chromium- to the root.
  4. When Reporting/Screenshotting attempts to initialize at Kibana setup, it uses the default data directory rather than the custom one set in kibana.yml

Analysis

I'm using a docker-compose based on the documentation and running the Kibana service by referencing a Dockerfile to build. NOTE: the OS here is Ubuntu 20, not RHEL 7, and the processor type is aarch64

kibana/Dockerfile:

FROM docker.elastic.co/kibana/kibana:7.17.17
USER root
RUN chown -R root:kibana /usr/share/kibana
RUN chmod -R 755 /usr/share/kibana
RUN mkdir -p /var/run/kibana /var/lib/kibana
RUN chown -R kibana:kibana /var/run/kibana /var/lib/kibana
USER kibana

kibana.yml

xpack.reporting.capture.browser.chromium.disableSandbox: false
pid.file: "/var/run/kibana/kibana.pid"
path.data: "/var/lib/kibana"

startup logs:

esd-kibana-1  | [2024-01-23T19:24:56.249+00:00][INFO ][plugins.reporting.chromium] Browser executable: /usr/share/kibana/x-pack/plugins/reporting/chromium/headless_shell-linux_arm64/headless_shell
esd-kibana-1  | [2024-01-23T19:24:56.250+00:00][ERROR][plugins.reporting] Error: EACCES: permission denied, mkdtemp '/usr/share/kibana/data/chromium-XXXXXX'
esd-kibana-1  |     at Object.mkdtempSync (node:fs:2960:18)
esd-kibana-1  |     at new HeadlessChromiumDriverFactory (/usr/share/kibana/x-pack/plugins/reporting/server/browsers/chromium/driver_factory/index.js:76:36)
esd-kibana-1  |     at Object.createDriverFactory (/usr/share/kibana/x-pack/plugins/reporting/server/browsers/chromium/index.js:25:54)
esd-kibana-1  |     at initializeBrowserDriverFactory (/usr/share/kibana/x-pack/plugins/reporting/server/browsers/index.js:43:29)
esd-kibana-1  |     at processTicksAndRejections (node:internal/process/task_queues:95:5)
esd-kibana-1  |     at /usr/share/kibana/x-pack/plugins/reporting/server/plugin.js:101:32

These logs are an effect of Reporting running initializing the headless driver by determining it's launch arguments and logging any relevant warnings. One of the launch arguments is the data path. This is provided by the Kibana platform by using the @kbn/utils package. Source. Note that the userDataDir value does not depend on the value of the disableSandbox option.

Compare to 8.12 startup logs:

esd-kibana-1  | [2024-01-23T19:49:42.256+00:00][INFO ][plugins.screenshotting.chromium] Browser executable: /usr/share/kibana/node_modules/@kbn/screenshotting-plugin/chromium/headless_shell-linux_arm64/headless_shell
esd-kibana-1  | [2024-01-23T19:49:42.257+00:00][ERROR][plugins.screenshotting] Error in screenshotting setup, it may not function properly.
esd-kibana-1  | [2024-01-23T19:49:42.257+00:00][ERROR][plugins.screenshotting] Error: EACCES: permission denied, mkdtemp '/usr/share/kibana/data/chromium-XXXXXX'
esd-kibana-1  |     at Object.mkdtempSync (node:fs:2919:3)
esd-kibana-1  |     at new HeadlessChromiumDriverFactory (/usr/share/kibana/node_modules/@kbn/screenshotting-plugin/server/browsers/chromium/driver_factory/index.js:62:36)
esd-kibana-1  |     at /usr/share/kibana/node_modules/@kbn/screenshotting-plugin/server/plugin.js:50:14
esd-kibana-1  |     at processTicksAndRejections (node:internal/process/task_queues:95:5)

At this point, it seems the bug behavior is not related to the sandbox option, or to Chromium. The code depends on the @kbn/utils package's getDataPath function's correctness.

@Ikuni17 @jbudz @mistic it seems this bug really is in the @kbn/utils domain, and not the reporting/screenshotting domain. Is there another issue this can be closed in favor of?

Ikuni17 commented 9 months ago

@tsullivan you're correct that this a @kbn/utils issue, specifically getDataPath. We don't have another issue for this, I would say we can remove the SharedUX team from this and OPs will handle it.

bm1216 commented 4 months ago

Hey, any progress with this issue? Maybe there's a solution that I'm overlooking

Ikuni17 commented 4 months ago

@bm1216 I haven't had a chance to revisit this. We have merged this twice with various fixes, but it creates issues downstream for other services. So, we'll have to handle this more gracefully and/or notify the downstream services of the change beforehand.