search

elastic / kibana

Your window into the Elastic Stack
https://www.elastic.co/products/kibana
Other
19.72k stars 8.14k forks source link

Fleet API package_policies (apm integration): misleading `Cannot read properties of undefined (reading 'map')` #154743

Open markush81 opened 1 year ago

markush81 commented 1 year ago

Kibana version:

8.7.0

Elasticsearch version:

8.7.0

Server OS version:

docker container exec -it kibana bash
kibana@kibana:~$ uname -a
Linux kibana 5.15.49-linuxkit #1 SMP PREEMPT Tue Sep 13 07:51:32 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux

Browser version:

curl --version
curl 7.87.0 (x86_64-apple-darwin22.0) libcurl/7.87.0 (SecureTransport) LibreSSL/3.3.6 zlib/1.2.11 nghttp2/1.51.0

Browser OS version:

macOS 13

Original install method (e.g. download page, yum, from source, etc.):

Docker (docker.elastic.co/kibana/kibana:8.7.0)

Describe the bug:

Updating APM package_policy via API fails.

Steps to reproduce:

  1. create agent policy
  2. add apm integration
  3. retrieve apm package policy id via api (because of https://github.com/elastic/kibana/issues/153205)
 curl -s -u elastic-H "Content-Type: application/json" -H "kbn-xsrf: true" http://localhost:5601/api/fleet/package_policies | jq -c '.items[] | select (.name=="apm") | .id'
"f478272e-13d4-4f50-9bc1-289cfd7e26da"
  1. update apm package policy
curl -s -X PUT -u elastic-H "Content-Type: application/json" -H "kbn-xsrf: true" http://localhost:5601/api/fleet/package_policies/f478272e-13d4-4f50-9bc1-289cfd7e26da -d @apm.json
{
    "name": "apm",
    "namespace": "default",
    "package": {
        "name": "apm",
        "version": "8.7.0"
    },
    "enabled": true,
    "policy_id": "apm",
    "inputs": [
        {
            "type": "apm",
            "enabled": true,
            "vars": {
                "host": {
                    "value": "0.0.0.0:8200"
                },
                "url": {
                    "value": "http://localhost:8200"
                }
            }
        }
    ]
}

Expected behavior:

Updated package_policy

Screenshots (if relevant):

Errors in browser console (if relevant):

{"statusCode":500,"error":"Internal Server Error","message":"Cannot read properties of undefined (reading 'map')"}

Provide logs and/or server output (if relevant):

[2023-04-11T16:12:59.752+00:00][ERROR][plugins.fleet] An error occurred executing "packagePolicyUpdate" callback: Error: [package.title]: expected value of type [string] but got [undefined]
[2023-04-11T16:12:59.752+00:00][ERROR][plugins.fleet] Error: [package.title]: expected value of type [string] but got [undefined]
    at ObjectType.validate (/usr/share/kibana/node_modules/@kbn/config-schema/src/types/type.js:84:13)
    at PackagePolicyClientImpl.runExternalCallbacks (/usr/share/kibana/x-pack/plugins/fleet/server/services/package_policy.js:946:67)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at PackagePolicyClientImpl.update (/usr/share/kibana/x-pack/plugins/fleet/server/services/package_policy.js:349:31)
    at updatePackagePolicyHandler (/usr/share/kibana/x-pack/plugins/fleet/server/routes/package_policy/handlers.js:331:34)
    at Router.handle (/usr/share/kibana/node_modules/@kbn/core-http-router-server-internal/src/router.js:141:30)
    at handler (/usr/share/kibana/node_modules/@kbn/core-http-router-server-internal/src/router.js:107:50)
    at exports.Manager.execute (/usr/share/kibana/node_modules/@hapi/hapi/lib/toolkit.js:60:28)
    at Object.internals.handler (/usr/share/kibana/node_modules/@hapi/hapi/lib/handler.js:46:20)
    at exports.execute (/usr/share/kibana/node_modules/@hapi/hapi/lib/handler.js:31:20)
    at Request._lifecycle (/usr/share/kibana/node_modules/@hapi/hapi/lib/request.js:371:32)
    at Request._execute (/usr/share/kibana/node_modules/@hapi/hapi/lib/request.js:281:9)
[2023-04-11T16:12:59.756+00:00][ERROR][plugins.fleet] TypeError: Cannot read properties of undefined (reading 'map')
    at assignStreamIdToInput (/usr/share/kibana/x-pack/plugins/fleet/server/services/package_policy.js:1064:28)
    at /usr/share/kibana/x-pack/plugins/fleet/server/services/package_policy.js:382:58
    at Array.map (<anonymous>)
    at PackagePolicyClientImpl.update (/usr/share/kibana/x-pack/plugins/fleet/server/services/package_policy.js:382:45)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at updatePackagePolicyHandler (/usr/share/kibana/x-pack/plugins/fleet/server/routes/package_policy/handlers.js:331:34)
    at Router.handle (/usr/share/kibana/node_modules/@kbn/core-http-router-server-internal/src/router.js:141:30)
    at handler (/usr/share/kibana/node_modules/@kbn/core-http-router-server-internal/src/router.js:107:50)
    at exports.Manager.execute (/usr/share/kibana/node_modules/@hapi/hapi/lib/toolkit.js:60:28)
    at Object.internals.handler (/usr/share/kibana/node_modules/@hapi/hapi/lib/handler.js:46:20)
    at exports.execute (/usr/share/kibana/node_modules/@hapi/hapi/lib/handler.js:31:20)
    at Request._lifecycle (/usr/share/kibana/node_modules/@hapi/hapi/lib/request.js:371:32)
    at Request._execute (/usr/share/kibana/node_modules/@hapi/hapi/lib/request.js:281:9)

I am not sure if the error about package.title is related, but even adding title to package sub-object fails (which is expeced, according to https://petstore.swagger.io/?url=https://raw.githubusercontent.com/elastic/kibana/8.7/x-pack/plugins/fleet/common/openapi/bundled.json)

Any additional context:

markush81 commented 1 year ago

The trick seems to be, to add: "streams": [], the error does not indicate that.

I found out while trying to reproduce this all with version 8.6.2 and there it behaved differently:

...
    "inputs": [
        {
            "type": "apm",
            "enabled": true,
            "streams": [], <----
            "vars": {
                "host": {
                    "value": "0.0.0.0:8200"
                },
                "url": {
                    "value": "http://localhost:8200"
                }
            }
        }
    ]
...

in 8.7.0 ... works to create it, but is unusable. Usually it was like adding just the important variables within PUT is enough and rest is defaults, but it seems not to be the case anymore.

So the real problem is: giving a proper error message to the API user.

elasticmachine commented 1 year ago

Pinging @elastic/fleet (Team:Fleet)

markush81 commented 1 year ago

Maybe some more background: it is unclear to me how to create a good automated setup of fleet - without the need to always POST full property sets.

The pre-configured variant i also tried, but there changes do not reflect (also not with is_managed: true- maybe another bug).