`TypeError: Cannot convert undefined or null to object` from elastic-apm-http-client

anotheri commented 4 years ago

Describe the bug

APM server doesn't show tracked transactions data, although all the logs in node application (even with logger level trace) show that everything's ok. After some investigation and enabling logUncaughtExceptions: true I've noticed the next error: TypeError: Cannot convert undefined or null to object from elastic-apm-http-client module (see the full error trace log in "additional context" section below). As far as i've got it happens when it tries to send cache/redis transaction tracked from connect-redis dependency used as session storage. To fix (skip) the issue i've added filter like this if (payload.type === 'cache') return false;.

Environment (please complete the following information)

OS: Linux, docker container: "node:12-stretch"
Node.js version: 12
APM Server version: 7.3.1
Agent version: 3.1.0

How are you starting the agent? (please tick one of the boxes)

[x] Calling agent.start() directly (e.g. require('elastic-apm-node').start(...))
[ ] Requiring elastic-apm-node/start from within the source code
[ ] Starting node with -r elastic-apm-node/start

Additional context

TypeError: Cannot convert undefined or null to object

Click to expand error trace
``` app_1 | TypeError: Cannot convert undefined or null to object app_1 | at Function.keys () app_1 | at truncateCustomKeys (/opt/app/node_modules/elastic-apm-http-client/lib/truncate.js:309:23) app_1 | at truncateCustomKeys (/opt/app/node_modules/elastic-apm-http-client/lib/truncate.js:316:22) app_1 | at Object.truncSpan [as span] (/opt/app/node_modules/elastic-apm-http-client/lib/truncate.js:136:12) app_1 | at Client._encode (/opt/app/node_modules/elastic-apm-http-client/index.js:327:27) app_1 | at Client.encodeObject (/opt/app/node_modules/elastic-apm-http-client/index.js:272:15) app_1 | at Array.map () app_1 | at Client._writevCleaned (/opt/app/node_modules/elastic-apm-http-client/index.js:276:22) app_1 | at processBatch (/opt/app/node_modules/elastic-apm-http-client/index.js:250:12) app_1 | at Client._writev (/opt/app/node_modules/elastic-apm-http-client/index.js:268:3) app_1 | at doWrite (/opt/app/node_modules/elastic-apm-http-client/node_modules/readable-stream/lib/_stream_writable.js:405:97) app_1 | at clearBuffer (/opt/app/node_modules/elastic-apm-http-client/node_modules/readable-stream/lib/_stream_writable.js:501:5) app_1 | at Client.Writable.uncork (/opt/app/node_modules/elastic-apm-http-client/node_modules/readable-stream/lib/_stream_writable.js:317:94) app_1 | at Timeout._onTimeout (/opt/app/node_modules/elastic-apm-http-client/index.js:299:14) app_1 | at listOnTimeout (internal/timers.js:531:17) app_1 | at processTimers (internal/timers.js:475:7) ```
Agent config options

Click to expand
``` { logUncaughtExceptions: true } ```
package.json dependencies:

Click to expand
``` "@google/maps": "^0.4.6", "@sindresorhus/slugify": "^0.9.1", "async": "^2.1.4", "aws-sdk": "^2.83.0", "axios": "^0.16.2", "basic-auth": "^1.0.0", "best": "git://github.com/clevertech/Best.git#v0.1.1", "bluebird": "^3.4.7", "body-parser": "^1.16.0", "bootable": "^0.2.4", "botkit": "^0.2.2", "bower": "^1.8.4", "compression": "^1.6.2", "connect-ensure-login": "^0.1.1", "connect-flash": "^0.1.1", "connect-history-api-fallback": "^1.6.0", "connect-redis": "^4.0.3", "cookie-parser": "^1.1.0", "cron": "1.5.1", "crypto": "^0.0.3", "csurf": "^1.9.0", "dedent-js": "^1.0.1", "dragula": "^3.7.2", "dropzone": "4.0.1", "elastic-apm-node": "^3.1.0", "elasticsearch": "^13.3.1", "electrolyte": "0.3.0", "element-ui": "2.10.0", "errorhandler": "^1.5.0", "express": "4.16.4", "express-csv": "0.6.0", "express-paginate": "^0.2.2", "express-session": "^1.15.0", "font-awesome": "4.5.0", "fs-finder": "^1.8.1", "github": "^5.2.2", "googleapis": "^2.1.6", "hat": "0.0.3", "helmet": "^3.4.0", "http-aws-es": "^2.0.5", "jade": "^1.3.1", "jsondiffpatch": "^0.2.4", "juice": "^4.0.2", "knox-s3": "^0.9.5", "latinize": "^0.4.0", "less": "^2.5.1", "lodash": "^4.17.4", "luxon": "^1.16.0", "marked": "^0.3.6", "merge-defaults": "^0.2.1", "mime": "^1.3.4", "mkdirp": "^0.5.1", "moment": "^2.18.0", "moment-timezone": "0.5.11", "money": "0.2.x", "mongoose": "4.7.5", "mongoose-auto-increment": "^5.0.1", "mongoose-deep-populate": "^3.0.0", "mongoose-delete": "^0.5.0", "mongoose-json-select": "^0.2.1", "mongoose-latinize": "^0.1.2", "mongoose-paginate": "^5.0.3", "morris.js": "https://github.com/morrisjs/morris.js.git", "needle": "^0.8.1", "nock": "^10.0.6", "node-csv": "^0.1.2", "nodemailer": "^3.1.8", "nodemailer-mandrill-transport": "1.2.0", "passport": "^0.3.2", "passport-google-oauth": "^0.1.5", "passport-google-token": "^0.1.0", "passport-local-mongoose": "^1.0.0", "passport-oauth": "^1.0.0", "passport-oauth2": "^1.1.2", "passport-openidconnect": "^0.0.2", "pdfkit": "^0.7.1", "phantom": "2.1.21", "pluralize": "^4.0.0", "querystring": "^0.2.0", "randomstring-extended": "^1.0.0", "raphael": "2.2.0", "raven": "^1.2.0", "redis": "2.8.0", "redis-mock": "^0.46.0", "request": "^2.88.0", "response-time": "^2.3.2", "rome": "2.1.22", "serve-favicon": "^2.4.1", "serve-static": "^1.12.1", "shopify-api-node": "^2.19.3", "skipper": "^0.5.9", "skipper-s3": "^0.5.9", "slack": "^8.1.0", "slack-winston": "0.0.1", "socket.io": "^2.1.1", "socket.io-redis": "^5.2.0", "sortablejs": "1.7.0", "strength": "^0.1.4", "sweetalert2": "0.4.5", "timerish": "^1.0.0", "twilio": "^2.2.1", "twix": "^1.1.4", "tz-lookup": "^6.1.24", "underscore": "1.8.3", "underscore.string": "^3.3.4", "v-clipboard": "^2.2.1", "validator": "^3.13.0", "vue": "2.5.16", "vue-datetime": "^1.0.0-beta.10", "vue-dplayer": "^0.0.10", "vue-moment": "4.0.0", "vue-mq": "^1.0.1", "vue-responsive-components": "^0.2.3", "vue-router": "3.0.2", "vuex": "2.3.1", "weekstart": "^1.0.0", "winston": "^2.3.1", "winston-mongodb": "^2.0.8", "winston-request-logger": "^1.0.7", "xml2js": "0.4.17" ```

paul-pagnan commented 4 years ago

Bump - I am also receiving this error. I have disabled redis using the disableInstrumentations: ['ioredis'] option in the apm.start which seems to have removed the error

I am on ioredis version 4.14.0 APM Agent version: 3.1.0 APM Server Version: Node Version: 12.12 OS: Mac Catalina 10.15.2 - Also occurs in Docker Ubuntu 18.10

trentm commented 2 years ago

I realize this is years after the fact, but I believe this was:

caused by https://github.com/elastic/apm-nodejs-http-client/pull/92/files#diff-0760ae25bbc618d8dcaf155372899ace802823e490f6b28f1716bb8b9abf6318R305-R309 (that change in the elastic-apm-http-client dep used by this agent would allow an attempt to Object.keys(null):

> Object.keys(null)
Uncaught TypeError: Cannot convert undefined or null to object
    at Function.keys (<anonymous>)

This was released as elastic-apm-http-client@9.1.0

Then was fixed shortly after in https://github.com/elastic/apm-nodejs-http-client/pull/93 and released as elastic-apm-http-client@9.1.1.

Apologies for the silence on this issue for so long and please re-open if you are hitting this same error with an agent (elastic-apm-node) that is at all recent.

elastic / apm-agent-nodejs

`TypeError: Cannot convert undefined or null to object` from elastic-apm-http-client #1512