search

elastic / kibana

Your window into the Elastic Stack
https://www.elastic.co/products/kibana
Other
19.64k stars 8.23k forks source link

[Encrypted Saved Objects] Multiple plugins experiencing errors with encrypted saved objects in 8.11.1 #171630

Closed kpollich closed 11 months ago

kpollich commented 11 months ago

Ref https://discuss.elastic.co/t/fleet-initial-error/347225/7

We're seeing various errors related to encrypted saved objects across multiple plugins in 8.11.1 including Fleet and Rules + Alerts.

Examples

During Fleet Setup

Fleet setup throws an error for each agent policy when we try to generate uninstall tokens.

image

This will result in Fleet Setup failing, Fleet being unavailable, or the uninstall tokens page being unavailable.

2023-11-16T13:35:52.319+00:00][INFO ][plugins.fleet] Beginning fleet setup
[2023-11-16T13:35:55.464+00:00][ERROR][plugins.encryptedSavedObjects] Failed to decrypt "token" attribute: Unsupported state or unable to authenticate data
[2023-11-16T13:35:55.465+00:00][ERROR][plugins.encryptedSavedObjects] Failed to decrypt "token" attribute: Unsupported state or unable to authenticate data
[2023-11-16T13:35:55.466+00:00][ERROR][plugins.encryptedSavedObjects] Failed to decrypt "token" attribute: Unsupported state or unable to authenticate data
[2023-11-16T13:35:55.466+00:00][ERROR][plugins.encryptedSavedObjects] Failed to decrypt "token" attribute: Unsupported state or unable to authenticate data
[2023-11-16T13:35:55.467+00:00][ERROR][plugins.encryptedSavedObjects] Failed to decrypt "token" attribute: Unsupported state or unable to authenticate data
[2023-11-16T13:35:55.467+00:00][ERROR][plugins.encryptedSavedObjects] Failed to decrypt "token" attribute: Unsupported state or unable to authenticate data
[2023-11-16T13:35:55.468+00:00][ERROR][plugins.encryptedSavedObjects] Failed to decrypt "token" attribute: Unsupported state or unable to authenticate data
[2023-11-16T13:35:55.468+00:00][ERROR][plugins.encryptedSavedObjects] Failed to decrypt "token" attribute: Unsupported state or unable to authenticate data
[2023-11-16T13:35:55.478+00:00][ERROR][plugins.encryptedSavedObjects] Failed to decrypt "token" attribute: Unsupported state or unable to authenticate data
[2023-11-16T13:35:55.479+00:00][ERROR][plugins.fleet] Uninstall Token is missing the token.

This error originates in Fleet setup here:

https://github.com/elastic/kibana/blob/3a58207b53d515b06e80cd1a1e828bc77e8181ca/x-pack/plugins/fleet/server/services/setup.ts#L167-L178

Specifically, the final error is thrown in this function:

https://github.com/elastic/kibana/blob/3a58207b53d515b06e80cd1a1e828bc77e8181ca/x-pack/plugins/fleet/server/services/security/uninstall_token_service/index.ts#L499-L503

The encrypted saved objects error is thrown here:

https://github.com/elastic/kibana/blob/3a58207b53d515b06e80cd1a1e828bc77e8181ca/x-pack/plugins/encrypted_saved_objects/server/crypto/encrypted_saved_objects_service.ts#L545-L557

In the Rules and Alerts Plugin

I don't have logs for this, but a discuss user provided this screenshot:

image

The error is likely identical to the Fleet issue, however the user was able to regenerate their rules' API keys in order to alleviate the problem.

Investigation

We've been able to reproduce a subset of these errors by doing the following

Open questions

elasticmachine commented 11 months ago

Pinging @elastic/security-defend-workflows (Team:Defend Workflows)

elasticmachine commented 11 months ago

Pinging @elastic/fleet (Team:Fleet)

kpollich commented 11 months ago

Ping @elastic/kibana-security (Sorry I don't know your team label)

kpollich commented 11 months ago

Posting a summary of my findings and what the great folks on the Kibana Security team, ECK team, and Security Defend Workflow teams have helped out with today over Slack.

https://github.com/elastic/kibana/blob/be46cceaa9eb2a2817d1f1775bcbaca43529e02c/x-pack/plugins/fleet/server/services/security/uninstall_token_service/index.ts#L193-L228

I've tried a few different ways to recreate this outside of an ECK environment without success, e.g.

I've tried the above steps going from 8.9.2 -> 8.11.1, 8.10.4 -> 8.11.1, and 8.9.2 -> 8.11.1 all without a reproduction of the errors.

Something else to note: the uninstall tokens seem to be the most common culprit for these error logs, but Fleet has another encrypted saved object code path that we haven't seen errors reported for yet:

https://github.com/elastic/kibana/blob/be46cceaa9eb2a2817d1f1775bcbaca43529e02c/x-pack/plugins/fleet/server/services/security/message_signing_service.ts#L55-L105

If these saved objects are encountering the same decryption errors as the uninstall tokens, we'd expect to see Failed to decrypt "passphrase" attribute: Unsupported state or unable to authenticate data errors during Fleet setup before the uninstall token errors.

Finally, I have reason to suspect this limited to ECK. Kibana does not implicitly set xpack.encryptedSavedObjects.encryptionKey in distributed versions of Kibana. So, unless a self-hosted user has explicitly set this value via kibana.yml and then rotated it improperly (see docs), they wouldn't see any kind of error like this. When this key is unset for distributable versions of Kibana, you'll see logs like this during Kibana startup (I copy/pasted these from my local cluster running production builds of Kibana + ES 8.11.1):

[2023-11-21T14:24:05.431-05:00][WARN ][plugins.encryptedSavedObjects] Saved objects encryption key is not set. This will severely limit Kibana functionality. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.

[2023-11-21T14:24:05.466-05:00][WARN ][plugins.alerting] APIs are disabled because the Encrypted Saved Objects plugin is missing encryption key. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.

# During Fleet setup, we warn on this exact case as well
[2023-11-21T14:24:23.660-05:00][WARN ][plugins.fleet] xpack.encryptedSavedObjects.encryptionKey is not configured, private key passphrase is being stored in plain text
[2023-11-21T14:24:23.665-05:00][WARN ][plugins.fleet] xpack.encryptedSavedObjects.encryptionKey is not configured, agent uninstall tokens are being stored in plain text

So it appears to me that something has potentially changed the value of the encryptionKey config property on ECK clusters in the 8.11.0 or 8.11.1 release. ECK generates a random value for the xpack.encryptedSavedObjects.encryptionKey here: https://github.com/elastic/cloud-on-k8s/blob/main/pkg/controller/kibana/config_settings.go#L251.

Is there any kind of trigger that would cause this random value to change during a stack upgrade? If a K8's namespace or secret or ConfigMap or something was blown away for some reason during the upgrade, would that cause a new encryption key to be randomly generated and break decryption like this? @elastic/cloud-k8s could you perhaps weigh in on this?

kpollich commented 11 months ago

I need to debug some local environment stuff with Kubernetes, but I'll try and reproduce the upgrade issue specifically in an ECK setup tomorrow morning my time (eastern US).

kpollich commented 11 months ago

I spun up an ECK deployment on my local machine this morning and performed a stack upgrade from 8.9.2 -> 8.11.1 without issue. The value of xpack.encryptedSavedObjects.encryptionKey was consistent across stack versions as expected.

So, my hunch that the ECK operator changed the encryption key seems to not be accurate, at least in this instance. If we can get some kibana logs from before and after the stack upgrade for one of the instances where this bug is happening, that'd be massively helpful in tracking down the cause.

Full findings below:

8.9.2 cluster

# Kibana logs w/ hashed encryption key. Key is generated randomly by ECK as it's not explicitly set in Kibana config
[2023-11-22T12:53:03.208+00:00][INFO ][plugins.encryptedSavedObjects] Hashed 'xpack.encryptedSavedObjects.encryptionKey' for this instance: BOKS+JGibEowGQ+xyWsAWQ06rlqQHgFs3OJqaw/THZg=

Created 3 agent policies via Fleet UI

GET kbn:/api/fleet/agent_policies

{
  "items": [
    {
      "id": "04010160-8939-11ee-aa3d-eb15ae9ef4d3",
      "name": "Agent Policy 3",
      "description": "",
      "namespace": "default",
      "monitoring_enabled": [
        "logs",
        "metrics"
      ],
      "inactivity_timeout": 1209600,
      "status": "active",
      "is_managed": false,
      "revision": 1,
      "updated_at": "2023-11-22T13:14:06.844Z",
      "updated_by": "elastic",
      "schema_version": "1.1.1",
      "is_protected": false,
      "agents": 0
    },
    {
      "id": "f845fe40-8936-11ee-aa3d-eb15ae9ef4d3",
      "name": "Agent Policy 2",
      "description": "",
      "namespace": "default",
      "monitoring_enabled": [
        "logs",
        "metrics"
      ],
      "inactivity_timeout": 1209600,
      "status": "active",
      "is_managed": false,
      "revision": 1,
      "updated_at": "2023-11-22T12:59:28.166Z",
      "updated_by": "elastic",
      "schema_version": "1.1.1",
      "is_protected": false,
      "agents": 0
    },
    {
      "id": "efe25190-8936-11ee-aa3d-eb15ae9ef4d3",
      "name": "Agent Policy 1",
      "description": "",
      "namespace": "default",
      "monitoring_enabled": [
        "logs",
        "metrics"
      ],
      "inactivity_timeout": 1209600,
      "status": "active",
      "is_managed": false,
      "revision": 1,
      "updated_at": "2023-11-22T12:59:14.090Z",
      "updated_by": "elastic",
      "schema_version": "1.1.1",
      "is_protected": false,
      "agents": 0
    }
  ],
  "total": 3,
  "page": 1,
  "perPage": 20
}

Uninstall tokens feature is not active, so no tokens exist (expected)

GET kbn:/api/fleet/uninstall_tokens
{
  "statusCode": 404,
  "error": "Not Found",
  "message": "Not Found"
}

GET .kibana_ingest/_search?q=type:fleet-uninstall-tokens
{
  "took": 9,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 0,
      "relation": "eq"
    },
    "max_score": null,
    "hits": []
  }
}

8.11.1 cluster

Everything worked as expected 🙁

[2023-11-22T13:20:25.641+00:00][INFO ][plugins.encryptedSavedObjects] Hashed 'xpack.encryptedSavedObjects.encryptionKey' for this instance: BOKS+JGibEowGQ+xyWsAWQ06rlqQHgFs3OJqaw/THZg=
GET kbn:/api/fleet/uninstall_tokens
{
  "items": [
    {
      "id": "f0f540d0-8939-11ee-a3d6-65b5ebccf826",
      "policy_id": "04010160-8939-11ee-aa3d-eb15ae9ef4d3",
      "created_at": "2023-11-22T13:20:44.380Z"
    },
    {
      "id": "f0f540d2-8939-11ee-a3d6-65b5ebccf826",
      "policy_id": "efe25190-8936-11ee-aa3d-eb15ae9ef4d3",
      "created_at": "2023-11-22T13:20:44.380Z"
    },
    {
      "id": "f0f540d1-8939-11ee-a3d6-65b5ebccf826",
      "policy_id": "f845fe40-8936-11ee-aa3d-eb15ae9ef4d3",
      "created_at": "2023-11-22T13:20:44.380Z"
    }
  ],
  "total": 3,
  "page": 1,
  "perPage": 20
}

GET .kibana_ingest/_search?q=type:fleet-uninstall-tokens
{
  "took": 9,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 3,
      "relation": "eq"
    },
    "max_score": 5.049856,
    "hits": [
      {
        "_index": ".kibana_ingest_8.9.2_001",
        "_id": "fleet-uninstall-tokens:f0f540d0-8939-11ee-a3d6-65b5ebccf826",
        "_score": 5.049856,
        "_source": {
          "fleet-uninstall-tokens": {
            "policy_id": "04010160-8939-11ee-aa3d-eb15ae9ef4d3",
            "token": "bLfvRRZgHaGrCTn1xoPQwe35iFcALvoPTRnJYQ30LVPzwWnSydysabBeYQPCRoNlu+35+0qbkhtH4PdcpzudMoFeHVt++ZO1S0x0Fq7lPyL6TEwJDpeQBiHO+NpAAG67EbMqS8P7doR8CZPfd8GbkLoyCILcEx260IfNAYea"
          },
          "type": "fleet-uninstall-tokens",
          "references": [],
          "managed": false,
          "coreMigrationVersion": "8.8.0",
          "updated_at": "2023-11-22T13:20:44.380Z",
          "created_at": "2023-11-22T13:20:44.380Z"
        }
      },
      {
        "_index": ".kibana_ingest_8.9.2_001",
        "_id": "fleet-uninstall-tokens:f0f540d2-8939-11ee-a3d6-65b5ebccf826",
        "_score": 5.049856,
        "_source": {
          "fleet-uninstall-tokens": {
            "policy_id": "efe25190-8936-11ee-aa3d-eb15ae9ef4d3",
            "token": "dAPbiKltWkcG5PAmnRWPSLgAEDIi2uMaxy1umTL35Z8qtwKmBDn8/Qu+nIZQett+gPASciI7r75/FQv6XiHgaXezrr89bUbPLzCPd1mWgfA853Ng5UYXuaEcCNg8dMpXrZX0lir7Rj5nRrdU+nYJXE1MNQ8XEv29NeFqFaIO"
          },
          "type": "fleet-uninstall-tokens",
          "references": [],
          "managed": false,
          "coreMigrationVersion": "8.8.0",
          "updated_at": "2023-11-22T13:20:44.380Z",
          "created_at": "2023-11-22T13:20:44.380Z"
        }
      },
      {
        "_index": ".kibana_ingest_8.9.2_001",
        "_id": "fleet-uninstall-tokens:f0f540d1-8939-11ee-a3d6-65b5ebccf826",
        "_score": 5.049856,
        "_source": {
          "fleet-uninstall-tokens": {
            "policy_id": "f845fe40-8936-11ee-aa3d-eb15ae9ef4d3",
            "token": "RrClbJ0IwCI8TuhSz4p0VahNgiSgJl+RSw7u0ZpLcDwmK/hgSAvLWHvDCGhMPXTdqyCgD+TXdPKRPFpD/Gs+VpSRN5SGOh8e58Bk2AbN1IG6+bgynRHISb2bZgYigFrnsugP3GgK6GGu9Nf+IKGgfNVpqM3DZLIzcQAiAooK"
          },
          "type": "fleet-uninstall-tokens",
          "references": [],
          "managed": false,
          "coreMigrationVersion": "8.8.0",
          "updated_at": "2023-11-22T13:20:44.380Z",
          "created_at": "2023-11-22T13:20:44.380Z"
        }
      }
    ]
  }
}
kpollich commented 11 months ago

I'll try an upgrade from 8.11.0 -> 8.11.1 to see if anything changes there, as the uninstall tokens will already exist during the stack upgrade process.

kpollich commented 11 months ago

Upgrade from 8.11.0 -> 8.11.1 shows the same behavior. Decryption works as expected, encryption key persists across stack upgrades

8.11.0 cluster

[2023-11-22T13:37:35.147+00:00][INFO ][plugins.encryptedSavedObjects] Hashed 'xpack.encryptedSavedObjects.encryptionKey' for this instance: AfazsxBuG/SEafuxGNFojYIEpCDGT62rwOLh2qaHYBE=
{
  "took": 10,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 3,
      "relation": "eq"
    },
    "max_score": 4.8225827,
    "hits": [
      {
        "_index": ".kibana_ingest_8.11.0_001",
        "_id": "fleet-uninstall-tokens:6793e4a0-893d-11ee-9258-11a7a4131f91",
        "_score": 4.8225827,
        "_source": {
          "fleet-uninstall-tokens": {
            "policy_id": "66f95250-893d-11ee-9258-11a7a4131f91",
            "token": "/Ob5GprdLaCu1pdW1HQyAQXruTmanG1dAfHOEyoXBiGrpGPA+kJd4Z4I8iKjJffsT93TVrvV+2eU4CjwQIEkr3AJAMs55KO/wQNP0T95qzU/OchZSobQulw81pZY7asYd3H3CzCGVFqysfEFZnxhNGthgoLQR2KNMZQy9OmT"
          },
          "type": "fleet-uninstall-tokens",
          "references": [],
          "managed": false,
          "coreMigrationVersion": "8.8.0",
          "updated_at": "2023-11-22T13:45:31.882Z",
          "created_at": "2023-11-22T13:45:31.882Z"
        }
      },
      {
        "_index": ".kibana_ingest_8.11.0_001",
        "_id": "fleet-uninstall-tokens:80a74400-893d-11ee-9258-11a7a4131f91",
        "_score": 4.8225827,
        "_source": {
          "fleet-uninstall-tokens": {
            "policy_id": "804044d0-893d-11ee-9258-11a7a4131f91",
            "token": "sG2QOoZMJSWDp5O18kN/76OMGHk+9C6gxTtYfynS/M/ktMC3w+42mKCDFnEtl8coTxCDE++MroGxZ/lUOal5h3mK0UllFcGZpzkotnLfiKXEaN/808XTtkBwG2iHzhhMiYzPitRxnwmXc1iQvVOl6hldoFAOSwVOoK1V8Osc"
          },
          "type": "fleet-uninstall-tokens",
          "references": [],
          "managed": false,
          "coreMigrationVersion": "8.8.0",
          "updated_at": "2023-11-22T13:46:13.952Z",
          "created_at": "2023-11-22T13:46:13.952Z"
        }
      },
      {
        "_index": ".kibana_ingest_8.11.0_001",
        "_id": "fleet-uninstall-tokens:8c994790-893d-11ee-9258-11a7a4131f91",
        "_score": 4.8225827,
        "_source": {
          "fleet-uninstall-tokens": {
            "policy_id": "8c45f770-893d-11ee-9258-11a7a4131f91",
            "token": "yoNIHVRpw6gE+1nKNa9VcHi9RvVMrecCjXx6kh+onKwcz7m6i3KqYrnGsU5bUVJ43dJPY79CDvQGadlC3NDZXDVagEIOXXgPwgdCns73Psd4sORYBMxLMC3IwoDfDOKMclBrX1TwL0Utjdn7WWXxacUDX4Bq1XEiSqiLRmWw"
          },
          "type": "fleet-uninstall-tokens",
          "references": [],
          "managed": false,
          "coreMigrationVersion": "8.8.0",
          "updated_at": "2023-11-22T13:46:33.993Z",
          "created_at": "2023-11-22T13:46:33.993Z"
        }
      }
    ]
  }
}

8.11.1

[2023-11-22T13:48:48.231+00:00][INFO ][plugins.encryptedSavedObjects] Hashed 'xpack.encryptedSavedObjects.encryptionKey' for this instance: AfazsxBuG/SEafuxGNFojYIEpCDGT62rwOLh2qaHYBE=
{
  "took": 3,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 3,
      "relation": "eq"
    },
    "max_score": 4.8225827,
    "hits": [
      {
        "_index": ".kibana_ingest_8.11.0_001",
        "_id": "fleet-uninstall-tokens:6793e4a0-893d-11ee-9258-11a7a4131f91",
        "_score": 4.8225827,
        "_source": {
          "fleet-uninstall-tokens": {
            "policy_id": "66f95250-893d-11ee-9258-11a7a4131f91",
            "token": "/Ob5GprdLaCu1pdW1HQyAQXruTmanG1dAfHOEyoXBiGrpGPA+kJd4Z4I8iKjJffsT93TVrvV+2eU4CjwQIEkr3AJAMs55KO/wQNP0T95qzU/OchZSobQulw81pZY7asYd3H3CzCGVFqysfEFZnxhNGthgoLQR2KNMZQy9OmT"
          },
          "type": "fleet-uninstall-tokens",
          "references": [],
          "managed": false,
          "coreMigrationVersion": "8.8.0",
          "updated_at": "2023-11-22T13:45:31.882Z",
          "created_at": "2023-11-22T13:45:31.882Z"
        }
      },
      {
        "_index": ".kibana_ingest_8.11.0_001",
        "_id": "fleet-uninstall-tokens:80a74400-893d-11ee-9258-11a7a4131f91",
        "_score": 4.8225827,
        "_source": {
          "fleet-uninstall-tokens": {
            "policy_id": "804044d0-893d-11ee-9258-11a7a4131f91",
            "token": "sG2QOoZMJSWDp5O18kN/76OMGHk+9C6gxTtYfynS/M/ktMC3w+42mKCDFnEtl8coTxCDE++MroGxZ/lUOal5h3mK0UllFcGZpzkotnLfiKXEaN/808XTtkBwG2iHzhhMiYzPitRxnwmXc1iQvVOl6hldoFAOSwVOoK1V8Osc"
          },
          "type": "fleet-uninstall-tokens",
          "references": [],
          "managed": false,
          "coreMigrationVersion": "8.8.0",
          "updated_at": "2023-11-22T13:46:13.952Z",
          "created_at": "2023-11-22T13:46:13.952Z"
        }
      },
      {
        "_index": ".kibana_ingest_8.11.0_001",
        "_id": "fleet-uninstall-tokens:8c994790-893d-11ee-9258-11a7a4131f91",
        "_score": 4.8225827,
        "_source": {
          "fleet-uninstall-tokens": {
            "policy_id": "8c45f770-893d-11ee-9258-11a7a4131f91",
            "token": "yoNIHVRpw6gE+1nKNa9VcHi9RvVMrecCjXx6kh+onKwcz7m6i3KqYrnGsU5bUVJ43dJPY79CDvQGadlC3NDZXDVagEIOXXgPwgdCns73Psd4sORYBMxLMC3IwoDfDOKMclBrX1TwL0Utjdn7WWXxacUDX4Bq1XEiSqiLRmWw"
          },
          "type": "fleet-uninstall-tokens",
          "references": [],
          "managed": false,
          "coreMigrationVersion": "8.8.0",
          "updated_at": "2023-11-22T13:46:33.993Z",
          "created_at": "2023-11-22T13:46:33.993Z"
        }
      }
    ]
  }
}
kpollich commented 11 months ago

We've validated a workaround that will unblock Fleet:

To unblock Fleet, you'll need to blow away and regenerate all uninstall tokens for your agent policies. It's recommended to keep a record of the existing uninstall tokens somewhere safe, in case they are present elsewhere in your environment or eventually appear in logs when an agent is uninstalled.

Perform the following actions, all requests are expected to be made in Kibana dev tools:

# Create a role with CRUD access to system indices
POST _security/role/system-index-superuser
{
  "indices": [
    {
      "names": [
        "*"
      ],
      "privileges": [
        "all"
      ],
      "allow_restricted_indices": true
    }
  ]
}

# Create a user and assign it that role
POST _security/user/temp_user
{
  "password": "temp_password",
  "roles": [
    "superuser",
    "system-index-superuser"
  ]
}

Then, in another browser session (incognito/private window is easiest) log into Kibana as that user and open dev tools while logged in as the temporary user with elevated permissions.

# Check the list of Fleet uninstall tokens, note the results somewhere like a .txt file
GET .kibana_ingest/_search?q=type:fleet-uninstall-tokens

# Delete all uninstall tokens
POST .kibana_ingest/_delete_by_query
{
  "query": {
    "match": { 
      "type": "fleet-uninstall-tokens"
    }
  }
}

# Rerun Fleet setup to generate new tokens
POST kbn:/api/fleet/setup

# Fetch the new tokens and compare them against the previous results to make sure new
# values have been generated
GET .kibana_ingest/_search?q=type:fleet-uninstall-tokens

Afterwards, the temp_user can be deleted from the initial Kibana browser session.

DELETE _security/user/temp_user

We're also working on an fix for 8.11.2 that will prevent Fleet from being blocked by encryption errors and improve the error logging for the uninstall token service to see if we can get a better idea of what's happening here: https://github.com/elastic/kibana/pull/171998.

The root cause is still unclear, and we're continuing to investigate.

kpollich commented 11 months ago

Hi all. The security team is tracking this internally, and the workaround above should unblock anyone that runs into the issue as described. I'm closing this in favor of the internal issue in the security team's repo.

huangharry commented 10 months ago

Hi Kpollich, Does it fixed on ELK 8.11.2 or 8.11.3?

kpollich commented 10 months ago

Hi @huangharry - if you're experiencing issues with Fleet specifically the workaround in this comment should unblock you.

Are you experiencing issues related to encrypted saved objects in other plugins? Happy to take a look at any error logs, screenshots, or other information you could provide 🙏