search

open-telemetry / opentelemetry-js-contrib

OpenTelemetry instrumentation for JavaScript modules
https://opentelemetry.io
Apache License 2.0
697 stars 513 forks source link

get TAV tests passing again #1860

Closed trentm closed 10 months ago

trentm commented 10 months ago

This is a working issue to attempt to get test-all-versions (TAV) tests passing again. They had been failing for a long time due to https://github.com/open-telemetry/opentelemetry-js-contrib/issues/1477 and other issues.

current status

https://github.com/open-telemetry/opentelemetry-js-contrib/actions/runs/7174025293/job/19534639796 is a run after some aws-sdk and cucumber TAV fixes were merged. The Node 16 run actually passed! The jobs for Node.js 14 and 18 failed. I don't yet know why. They all took ~1h40m.

Update (2023-12-20):

timings from logs

I downloaded the logs archive (122MB) and unzipped to "~/Downloads/tav-pass". This grep roughly shows the runtimes and failures for each TAV module:

[~/Downloads/tav-pass/tav  Run test-all-versions (18)]
% rg -N test-all-versions "9_Run test-all-versions.txt"
2023-12-11T22:24:50.3355392Z ##[group]Run npm run --if-present --workspaces test-all-versions
2023-12-11T22:24:50.3355999Z npm run --if-present --workspaces test-all-versions
2023-12-11T22:24:50.6565829Z > @opentelemetry/instrumentation-amqplib@0.33.4 test-all-versions
2023-12-11T22:26:25.7835648Z > @opentelemetry/instrumentation-cucumber@0.2.0 test-all-versions
2023-12-11T22:31:47.1645628Z > @opentelemetry/instrumentation-dataloader@0.5.3 test-all-versions
2023-12-11T22:32:25.7075321Z > @opentelemetry/instrumentation-lru-memoizer@0.33.4 test-all-versions
2023-12-11T22:35:12.7549151Z > @opentelemetry/instrumentation-mongoose@0.33.4 test-all-versions
2023-12-11T23:05:02.9052204Z > @opentelemetry/instrumentation-socket.io@0.34.4 test-all-versions
2023-12-11T23:07:06.9115988Z > @opentelemetry/instrumentation-tedious@0.6.4 test-all-versions
2023-12-11T23:11:40.6308502Z > @opentelemetry/instrumentation-aws-sdk@0.37.1 test-all-versions
2023-12-11T23:18:08.6949130Z > @opentelemetry/instrumentation-bunyan@0.34.0 test-all-versions
2023-12-11T23:18:40.1813771Z > @opentelemetry/instrumentation-express@0.34.0 test-all-versions
2023-12-11T23:20:09.7172730Z > @opentelemetry/instrumentation-fastify@0.32.5 test-all-versions
2023-12-11T23:20:27.8149905Z npm ERR! Lifecycle script `test-all-versions` failed with error:
2023-12-11T23:20:27.8163184Z > @opentelemetry/instrumentation-graphql@0.36.0 test-all-versions
2023-12-11T23:22:43.1915498Z > @opentelemetry/instrumentation-ioredis@0.36.0 test-all-versions
2023-12-11T23:25:03.5036879Z > @opentelemetry/instrumentation-koa@0.36.3 test-all-versions
2023-12-11T23:28:36.7556491Z > @opentelemetry/instrumentation-mongodb@0.38.0 test-all-versions
2023-12-11T23:40:38.9141329Z > @opentelemetry/instrumentation-mysql2@0.34.4 test-all-versions
2023-12-11T23:45:43.4366368Z > @opentelemetry/instrumentation-nestjs-core@0.33.3 test-all-versions
2023-12-11T23:50:08.8430844Z > @opentelemetry/instrumentation-pg@0.37.1 test-all-versions
2023-12-11T23:51:03.9225893Z > @opentelemetry/instrumentation-pino@0.34.4 test-all-versions
2023-12-11T23:51:13.7991792Z npm ERR! Lifecycle script `test-all-versions` failed with error:
2023-12-11T23:51:13.8000664Z > @opentelemetry/instrumentation-redis@0.35.4 test-all-versions
2023-12-11T23:53:09.5370372Z > @opentelemetry/instrumentation-redis-4@0.35.5 test-all-versions
2023-12-11T23:53:19.4790144Z npm ERR! Lifecycle script `test-all-versions` failed with error:
2023-12-11T23:53:19.4797764Z > @opentelemetry/instrumentation-restify@0.34.2 test-all-versions
2023-12-11T23:53:36.3773335Z > @opentelemetry/instrumentation-winston@0.33.0 test-all-versions
  1. There are failures in fastify, pino and redis-4 to look into.
  2. Timings (by visually comparing the timestamps) suggest mongoose (30 minutes), and maybe mongodb (12 minutes) are the best candidates for improvement.
Timings details ``` 2023-12-11T22:24:50.6565829Z > @opentelemetry/instrumentation-amqplib@0.33.4 test-all-versions <2min 2023-12-11T22:26:25.7835648Z > @opentelemetry/instrumentation-cucumber@0.2.0 test-all-versions ~5min 2023-12-11T22:31:47.1645628Z > @opentelemetry/instrumentation-dataloader@0.5.3 test-all-versions <1min 2023-12-11T22:32:25.7075321Z > @opentelemetry/instrumentation-lru-memoizer@0.33.4 test-all-versions <3min 2023-12-11T22:35:12.7549151Z > @opentelemetry/instrumentation-mongoose@0.33.4 test-all-versions 30min! 2023-12-11T23:05:02.9052204Z > @opentelemetry/instrumentation-socket.io@0.34.4 test-all-versions 2min 2023-12-11T23:07:06.9115988Z > @opentelemetry/instrumentation-tedious@0.6.4 test-all-versions 4-5min 2023-12-11T23:11:40.6308502Z > @opentelemetry/instrumentation-aws-sdk@0.37.1 test-all-versions ~6min 2023-12-11T23:18:08.6949130Z > @opentelemetry/instrumentation-bunyan@0.34.0 test-all-versions ~30s 2023-12-11T23:18:40.1813771Z > @opentelemetry/instrumentation-express@0.34.0 test-all-versions <2min 2023-12-11T23:20:09.7172730Z > @opentelemetry/instrumentation-fastify@0.32.5 test-all-versions 2023-12-11T23:20:27.8149905Z npm ERR! Lifecycle script `test-all-versions` failed with error: 2023-12-11T23:20:27.8163184Z > @opentelemetry/instrumentation-graphql@0.36.0 test-all-versions ~2min 2023-12-11T23:22:43.1915498Z > @opentelemetry/instrumentation-ioredis@0.36.0 test-all-versions 2-3min 2023-12-11T23:25:03.5036879Z > @opentelemetry/instrumentation-koa@0.36.3 test-all-versions 3-4min 2023-12-11T23:28:36.7556491Z > @opentelemetry/instrumentation-mongodb@0.38.0 test-all-versions 12min 2023-12-11T23:40:38.9141329Z > @opentelemetry/instrumentation-mysql2@0.34.4 test-all-versions 5min 2023-12-11T23:45:43.4366368Z > @opentelemetry/instrumentation-nestjs-core@0.33.3 test-all-versions 5min 2023-12-11T23:50:08.8430844Z > @opentelemetry/instrumentation-pg@0.37.1 test-all-versions 1min 2023-12-11T23:51:03.9225893Z > @opentelemetry/instrumentation-pino@0.34.4 test-all-versions 2023-12-11T23:51:13.7991792Z npm ERR! Lifecycle script `test-all-versions` failed with error: 2023-12-11T23:51:13.8000664Z > @opentelemetry/instrumentation-redis@0.35.4 test-all-versions 2min 2023-12-11T23:53:09.5370372Z > @opentelemetry/instrumentation-redis-4@0.35.5 test-all-versions 2023-12-11T23:53:19.4790144Z npm ERR! Lifecycle script `test-all-versions` failed with error: 2023-12-11T23:53:19.4797764Z > @opentelemetry/instrumentation-restify@0.34.2 test-all-versions <20s (just 2 versions tested) 2023-12-11T23:53:36.3773335Z > @opentelemetry/instrumentation-winston@0.33.0 test-all-versions <1min 2023-12-11T23:54:15.1523459Z -- ok 2023-12-11T23:54:15.1761080Z ##[error]Process completed with exit code 1. ```

failures

This section lists each of the failures.

Tip: Can search for -- fatal in the TAV log output to find the failures.

fastify

Fastify TAV failure ``` 2023-12-11T23:20:17.7161067Z -- installing ["fastify@4.24.3"] 2023-12-11T23:20:23.9223894Z -- running test "npm run test" with fastify (env: {}) 2023-12-11T23:20:24.2622261Z 2023-12-11T23:20:24.2623649Z > @opentelemetry/instrumentation-fastify@0.32.5 test 2023-12-11T23:20:24.2624803Z > nyc ts-mocha -p tsconfig.json 'test/**/*.test.ts' 2023-12-11T23:20:24.2625329Z 2023-12-11T23:20:26.6165290Z 2023-12-11T23:20:26.6167959Z 2023-12-11T23:20:26.6176639Z fastify 2023-12-11T23:20:27.7091430Z ✓ should work with ESM usage (1085ms) 2023-12-11T23:20:27.7100059Z 1) should work with ESM usage 2023-12-11T23:20:27.7107984Z 2023-12-11T23:20:27.7110938Z 2023-12-11T23:20:27.7112472Z 1 passing (1s) 2023-12-11T23:20:27.7113096Z 1 failing 2023-12-11T23:20:27.7114794Z 2023-12-11T23:20:27.7128988Z 1) fastify 2023-12-11T23:20:27.7129446Z should work with ESM usage: 2023-12-11T23:20:27.7129823Z 2023-12-11T23:20:27.7130375Z Uncaught AssertionError [ERR_ASSERTION]: Expected values to be strictly equal: 2023-12-11T23:20:27.7131113Z 2023-12-11T23:20:27.7131277Z 2 !== 1 2023-12-11T23:20:27.7131473Z 2023-12-11T23:20:27.7131737Z + expected - actual 2023-12-11T23:20:27.7132032Z 2023-12-11T23:20:27.7132185Z -2 2023-12-11T23:20:27.7132503Z +1 2023-12-11T23:20:27.7132810Z 2023-12-11T23:20:27.7133421Z at Object.checkCollector (test/instrumentation.test.ts:555:16) 2023-12-11T23:20:27.7135757Z at done (/home/runner/work/opentelemetry-js-contrib/opentelemetry-js-contrib/packages/opentelemetry-test-utils/src/test-fixtures.ts:242:24) 2023-12-11T23:20:27.7137473Z at processTicksAndRejections (node:internal/process/task_queues:95:5) 2023-12-11T23:20:27.7137876Z ```

pino

pino TAV failure ``` 2023-12-11T23:51:04.5955184Z -- required packages ["pino@8.16.2"] 2023-12-11T23:51:04.5955954Z -- installing ["pino@8.16.2"] 2023-12-11T23:51:10.7847466Z -- running test "npm run test" with pino (env: {}) 2023-12-11T23:51:11.1220812Z 2023-12-11T23:51:11.1222086Z > @opentelemetry/instrumentation-pino@0.34.4 test 2023-12-11T23:51:11.1223171Z > nyc ts-mocha -p tsconfig.json 'test/**/*.test.ts' 2023-12-11T23:51:11.1223705Z 2023-12-11T23:51:12.8870685Z 2023-12-11T23:51:12.8874854Z 2023-12-11T23:51:12.8885035Z PinoInstrumentation 2023-12-11T23:51:13.7045501Z ✓ should work with ESM default import (783ms) 2023-12-11T23:51:13.7055636Z 1) should work with ESM default import 2023-12-11T23:51:13.7058450Z 2023-12-11T23:51:13.7062296Z 2023-12-11T23:51:13.7063832Z 1 passing (820ms) 2023-12-11T23:51:13.7064284Z 1 failing 2023-12-11T23:51:13.7066447Z 2023-12-11T23:51:13.7082080Z 1) PinoInstrumentation 2023-12-11T23:51:13.7083009Z should work with ESM default import: 2023-12-11T23:51:13.7083542Z 2023-12-11T23:51:13.7084064Z Uncaught AssertionError [ERR_ASSERTION]: Expected values to be strictly equal: 2023-12-11T23:51:13.7085072Z 2023-12-11T23:51:13.7085200Z 2 !== 1 2023-12-11T23:51:13.7085346Z 2023-12-11T23:51:13.7085615Z + expected - actual 2023-12-11T23:51:13.7085937Z 2023-12-11T23:51:13.7086135Z -2 2023-12-11T23:51:13.7086485Z +1 2023-12-11T23:51:13.7086718Z 2023-12-11T23:51:13.7087138Z at Object.checkResult (test/pino.test.ts:327:16) 2023-12-11T23:51:13.7089046Z at done (/home/runner/work/opentelemetry-js-contrib/opentelemetry-js-contrib/packages/opentelemetry-test-utils/src/test-fixtures.ts:239:24) 2023-12-11T23:51:13.7090617Z at ChildProcess.exithandler (node:child_process:414:7) 2023-12-11T23:51:13.7091311Z at ChildProcess.emit (node:events:517:28) 2023-12-11T23:51:13.7091859Z at maybeClose (node:internal/child_process:1098:16) 2023-12-11T23:51:13.7092645Z at Process.ChildProcess._handle.onexit (node:internal/child_process:303:5) ```

redis-4

redis-4 TAV failure ``` 2023-12-11T23:53:10.0867078Z -- required packages ["redis@4.6.11"] 2023-12-11T23:53:10.0868024Z -- installing ["redis@4.6.11"] 2023-12-11T23:53:16.6341652Z -- running test "npm run test" with redis (env: {}) 2023-12-11T23:53:16.9909066Z 2023-12-11T23:53:16.9910543Z > @opentelemetry/instrumentation-redis-4@0.35.5 test 2023-12-11T23:53:16.9912423Z > nyc ts-mocha -p tsconfig.json --require '@opentelemetry/contrib-test-utils' 'test/redis.test.ts' 2023-12-11T23:53:16.9913542Z 2023-12-11T23:53:19.2833843Z 2023-12-11T23:53:19.2837166Z 2023-12-11T23:53:19.2882931Z redis@^4.0.0 2023-12-11T23:53:19.2885078Z redis commands 2023-12-11T23:53:19.3026767Z ✓ simple set and get 2023-12-11T23:53:19.3066197Z ✓ send general command 2023-12-11T23:53:19.3105749Z ✓ command with error 2023-12-11T23:53:19.3111253Z client connect 2023-12-11T23:53:19.3144523Z ✓ produces a span 2023-12-11T23:53:19.3182802Z ✓ sets error status on connection failure 2023-12-11T23:53:19.3216589Z ✓ omits basic auth from DB_CONNECTION_STRING span attribute 2023-12-11T23:53:19.3260031Z 1) "after each" hook in "redis@^4.0.0" 2023-12-11T23:53:19.3324965Z ✓ omits user_pwd query parameter from DB_CONNECTION_STRING span attribute 2023-12-11T23:53:19.3328546Z multi (transactions) commands 2023-12-11T23:53:19.3367436Z ✓ multi commands 2023-12-11T23:53:19.3387266Z ✓ multi command with generic command 2023-12-11T23:53:19.3412102Z ✓ multi command with error 2023-12-11T23:53:19.3447888Z ✓ multi command that rejects 2023-12-11T23:53:19.3577681Z ✓ duration covers create until server response 2023-12-11T23:53:19.3608208Z ✓ response hook for multi commands 2023-12-11T23:53:19.3612452Z config 2023-12-11T23:53:19.3612848Z dbStatementSerializer 2023-12-11T23:53:19.3627819Z ✓ custom dbStatementSerializer 2023-12-11T23:53:19.3645958Z ✓ dbStatementSerializer throws 2023-12-11T23:53:19.3649219Z responseHook 2023-12-11T23:53:19.3663536Z ✓ valid response hook 2023-12-11T23:53:19.3679693Z ✓ responseHook throws 2023-12-11T23:53:19.3683373Z requireParentSpan 2023-12-11T23:53:19.3711500Z ✓ set to true 2023-12-11T23:53:19.3714800Z 2023-12-11T23:53:19.3720270Z 2023-12-11T23:53:19.3721825Z 18 passing (89ms) 2023-12-11T23:53:19.3722217Z 1 failing 2023-12-11T23:53:19.3722378Z 2023-12-11T23:53:19.3725104Z 1) redis@^4.0.0 2023-12-11T23:53:19.3725713Z "after each" hook in "redis@^4.0.0": 2023-12-11T23:53:19.3728724Z Error: done() called multiple times in hook of file /home/runner/work/opentelemetry-js-contrib/opentelemetry-js-contrib/plugins/node/opentelemetry-instrumentation-redis-4/test/redis.test.ts; in addition, done() received error: TypeError: Cannot read properties of undefined (reading 'destroy') 2023-12-11T23:53:19.3731904Z at RedisSocket._RedisSocket_connect (/home/runner/work/opentelemetry-js-contrib/opentelemetry-js-contrib/node_modules/@redis/client/dist/lib/client/socket.js:154:71) 2023-12-11T23:53:19.3733111Z at processTicksAndRejections (node:internal/process/task_queues:95:5) 2023-12-11T23:53:19.3734631Z at Commander.connect (/home/runner/work/opentelemetry-js-contrib/opentelemetry-js-contrib/node_modules/@redis/client/dist/lib/client/index.js:185:9) 2023-12-11T23:53:19.3736328Z at /home/runner/work/opentelemetry-js-contrib/opentelemetry-js-contrib/plugins/node/opentelemetry-instrumentation-redis-4/test/redis.test.ts:75:7 { 2023-12-11T23:53:19.3737177Z uncaught: true 2023-12-11T23:53:19.3737437Z } 2023-12-11T23:53:19.3737702Z at process.emit (node:events:517:28) 2023-12-11T23:53:19.3738627Z at processEmit (/home/runner/work/opentelemetry-js-contrib/opentelemetry-js-contrib/node_modules/signal-exit/index.js:199:34) 2023-12-11T23:53:19.3740019Z at process.emit (/home/runner/work/opentelemetry-js-contrib/opentelemetry-js-contrib/node_modules/source-map-support/source-map-support.js:516:21) 2023-12-11T23:53:19.3740989Z at process._fatalException (node:internal/process/execution:158:25) 2023-12-11T23:53:19.3741605Z at processPromiseRejections (node:internal/process/promises:288:13) 2023-12-11T23:53:19.3742372Z at processTicksAndRejections (node:internal/process/task_queues:96:32) ```

num versions

Here is a command that shows the number of versions of each module being tested by each of the ".tav.yml" files. In this example ttav2 is an alias to this change to test-all-versions to add a dry-run command and include the number of versions in verbose output: https://github.com/watson/test-all-versions/pull/12

Highlights:

  1. mongoose is teseting 204 versions. Let's reduce that.
  2. mongodb is testing 88 versions. Let's see if we can reduce that.
  3. other candidates for reducing the number of tested versions: cucumber, lru-memoizer, tedious, mysql2, nestjs-core, and redis-4
command output showing number of module versions ``` % cd opentelemetry-js-contrib/plugins/node % ls */.tav.yml | cut -d/ -f1 | while read d; do echo; echo "# $d"; (cd $d; ttav2 --verbose -n | grep 'package versions matching'); done # instrumentation-amqplib -- 12 package versions matching ">=0.5.5": 0.5.5, 0.5.6, 0.6.0, 0.7.0, 0.7.1, 0.8.0, 0.9.0, 0.9.1, 0.10.0, 0.10.1, 0.10.2, 0.10.3 # instrumentation-cucumber -- 2 package versions matching "^10.0.0": 10.0.0, 10.0.1 -- 33 package versions matching "^8.0.0 || ^9.0.0": 8.0.0, 8.1.0, 8.1.1, 8.1.2, 8.2.0, 8.2.1, 8.2.2, 8.3.0, 8.3.1, 8.4.0, 8.5.0, 8.5.1, 8.5.2, 8.5.3, 8.6.0, 8.7.0, 8.8.0, 8.9.0, 8.9.1, 8.10.0, 8.11.0, 8.11.1, 9.0.0, 9.0.1, 9.1.0, 9.1.1, 9.1.2, 9.2.0, 9.3.0, 9.4.0, 9.5.0, 9.5.1, 9.6.0 # instrumentation-dataloader -- 5 package versions matching "^2.0.0": 2.0.0, 2.1.0, 2.2.0, 2.2.1, 2.2.2 # instrumentation-lru-memoizer -- 23 package versions matching ">=1.3 <3": 1.3.0, 1.3.1, 1.4.0, 1.5.0, 1.6.0, 1.7.0, 1.8.0, 1.9.0, 1.10.0, 1.10.1, 1.11.0, 1.11.1, 1.11.2, 1.12.0, 1.13.0, 2.0.0, 2.0.1, 2.1.0, 2.1.1, 2.1.2, 2.1.3, 2.1.4, 2.2.0 # instrumentation-mongoose -- 204 package versions matching ">=5.9.7 <7": 5.9.7, 5.9.9, 5.9.10, 5.9.11, 5.9.12, 5.9.13, 5.9.14, 5.9.15, 5.9.16, 5.9.17, 5.9.18, 5.9.19, 5.9.20, 5.9.21, 5.9.22, 5.9.23, 5.9.24, 5.9.25, 5.9.26, 5.9.27, 5.9.28, 5.9.29, 5.10.0, 5.10.1, 5.10.2, 5.10.3, 5.10.4, 5.10.5, 5.10.6, 5.10.7, 5.10.8, 5.10.9, 5.10.10, 5.10.11, 5.10.12, 5.10.13, 5.10.14, 5.10.15, 5.10.16, 5.10.17, 5.10.18, 5.10.19, 5.11.0, 5.11.1, 5.11.2, 5.11.3, 5.11.4, 5.11.5, 5.11.6, 5.11.7, 5.11.8, 5.11.9, 5.11.10, 5.11.11, 5.11.12, 5.11.13, 5.11.14, 5.11.15, 5.11.16, 5.11.17, 5.11.18, 5.11.19, 5.11.20, 5.12.0, 5.12.1, 5.12.2, 5.12.3, 5.12.4, 5.12.5, 5.12.6, 5.12.7, 5.12.8, 5.12.9, 5.12.10, 5.12.11, 5.12.12, 5.12.13, 5.12.14, 5.12.15, 5.13.0, 5.13.1, 5.13.2, 5.13.3, 5.13.4, 5.13.5, 5.13.6, 5.13.7, 5.13.8, 6.0.0, 6.0.1, 6.0.2, 6.0.3, 6.0.4, 5.13.9, 6.0.5, 6.0.6, 6.0.7, 6.0.8, 6.0.9, 5.13.10, 6.0.10, 5.13.11, 6.0.11, 5.13.12, 6.0.12, 5.13.13, 6.0.13, 6.0.14, 6.0.15, 6.1.0, 6.1.1, 6.1.2, 6.1.3, 5.13.14, 6.1.4, 6.1.5, 6.1.6, 6.1.7, 6.1.8, 6.1.9, 6.1.10, 6.2.0, 6.2.1, 6.2.2, 6.2.3, 6.2.4, 6.2.5, 6.2.6, 6.2.7, 6.2.8, 6.2.9, 6.2.10, 6.2.11, 6.3.0, 6.3.1, 6.3.2, 6.3.3, 6.3.4, 6.3.5, 6.3.6, 6.3.7, 6.3.8, 6.3.9, 6.4.0, 6.4.1, 6.4.2, 6.4.3, 6.4.4, 6.4.5, 6.4.6, 6.4.7, 6.5.0, 6.5.1, 6.5.2, 5.13.15, 6.5.3, 6.5.4, 6.5.5, 6.6.0, 6.6.1, 6.6.2, 6.6.3, 6.6.4, 6.6.5, 6.6.6, 6.6.7, 6.7.0, 6.7.1, 6.7.2, 6.7.3, 6.7.4, 6.7.5, 6.8.0, 6.8.1, 6.8.2, 6.8.3, 6.8.4, 6.9.0, 6.9.1, 6.9.2, 5.13.16, 6.9.3, 6.10.0, 6.10.1, 6.10.2, 6.10.3, 6.10.4, 5.13.17, 6.10.5, 6.11.0, 6.11.1, 6.11.2, 5.13.18, 5.13.19, 6.11.3, 5.13.20, 6.11.4, 6.11.5, 6.11.6, 6.12.0, 6.12.1, 5.13.21, 6.12.2, 6.12.3 # instrumentation-socket.io -- 11 package versions matching "4.4.1 || 4.5.1 || 4.5.2 || ^4.5.3": 4.4.1, 4.5.1, 4.5.2, 4.5.3, 4.5.4, 4.6.0, 4.6.1, 4.6.2, 4.7.0, 4.7.1, 4.7.2 -- 0 package versions matching "^3.1.3": -- no versions of socket.io matching ^3.1.3 -- 5 package versions matching "2.1.1 || 2.2.0 || 2.3.0 || 2.4.0 || ^2.5.0": 2.1.1, 2.2.0, 2.3.0, 2.4.0, 2.5.0 # instrumentation-tedious -- 30 package versions matching "1.11.0 || 1.14.0 || 2.7.1 || 3.0.1 || 4.2.0 || ^6.7.0 || 8.3.0 || 9.2.3 || 11.0.9 || 11.2.0 || 11.4.0 || ^11.8.0 || ^12.3.0 || ^13.2.0 || ^14.0.0 || ^15.0.0": 1.11.0, 1.14.0, 2.7.1, 3.0.1, 4.2.0, 6.7.0, 8.3.0, 9.2.3, 11.0.9, 11.2.0, 11.4.0, 6.7.1, 11.8.0, 12.3.0, 13.2.0, 14.0.0, 14.1.0, 14.2.0, 14.3.0, 14.4.0, 14.5.0, 14.6.0, 14.6.1, 14.7.0, 15.0.0, 15.0.1, 15.1.0, 15.1.1, 15.1.2, 15.1.3 # opentelemetry-instrumentation-aws-sdk -- 10 package versions matching "3.24.0 || 3.85.0 || 3.194.0 || 3.278.0 || 3.357.0 || 3.461.0 || >=3.462.0": 3.24.0, 3.85.0, 3.194.0, 3.278.0, 3.357.0, 3.461.0, 3.462.0, 3.465.0, 3.468.0, 3.470.0 -- 10 package versions matching "3.6.1 || 3.53.0 || 3.163.0 || 3.266.0 || 3.354.0 || 3.458.0 || >=3.462.0": 3.6.1, 3.53.0, 3.163.0, 3.266.0, 3.354.0, 3.458.0, 3.462.0, 3.465.0, 3.468.0, 3.470.0 -- 15 package versions matching "2.308.0 || 2.548.0 || 2.785.0 || 2.1025.0 || 2.1265.0 || 2.1506.0 || >=2.1508.0": 2.308.0, 2.548.0, 2.785.0, 2.1025.0, 2.1265.0, 2.1506.0, 2.1508.0, 2.1509.0, 2.1510.0, 2.1511.0, 2.1512.0, 2.1513.0, 2.1514.0, 2.1515.0, 2.1516.0 # opentelemetry-instrumentation-bunyan -- 4 package versions matching "^1.8.12": 1.8.12, 1.8.13, 1.8.14, 1.8.15 # opentelemetry-instrumentation-express -- 10 package versions matching "^4.16.2": 4.16.2, 4.16.3, 4.16.4, 4.17.0, 4.17.1, 4.17.2, 4.17.3, 4.18.0, 4.18.1, 4.18.2 # opentelemetry-instrumentation-fastify -- 1 package versions matching "4.7.4": 4.7.4 -- 2 package versions matching "4.0.0 || >=4.24.3 <5": 4.0.0, 4.24.3 # opentelemetry-instrumentation-graphql -- 17 package versions matching "16.4.0 || 16.3.0 || 16.2.0 || 16.0.0 || ^15.8.0 || 15.7.2 || 15.6.1 || 15.6.0 || 15.5.3 || 15.5.1 || 15.5.0 || 15.4.0 || 15.3.0 || ^14.7.0 || 14.6.0 || 14.5.8 || 14.0.0": 14.0.0, 14.5.8, 14.6.0, 15.3.0, 14.7.0, 15.4.0, 15.5.0, 15.5.1, 15.5.3, 15.6.0, 15.6.1, 15.7.2, 16.0.0, 15.8.0, 16.2.0, 16.3.0, 16.4.0 # opentelemetry-instrumentation-ioredis -- 16 package versions matching "^2.5.0 || ^3.2.2 || 4.14.1 || 4.16.3 || 4.17.3 || 4.18.0 || 4.19.2 || 4.19.4 || 4.22.0 || 4.24.5 || 4.26.0 || 4.27.11 || ^4.28.5 || 5.0.6 || 5.2.6 || ^5.3.2": 2.5.0, 3.2.2, 4.14.1, 4.16.3, 4.17.3, 4.18.0, 4.19.2, 4.19.4, 4.22.0, 4.24.5, 4.26.0, 4.27.11, 4.28.5, 5.0.6, 5.2.6, 5.3.2 # opentelemetry-instrumentation-koa -- 17 package versions matching "^2.7.0": 2.7.0, 2.8.0, 2.8.1, 2.8.2, 2.9.0, 2.10.0, 2.11.0, 2.12.0, 2.12.1, 2.13.0, 2.13.1, 2.13.2, 2.13.3, 2.13.4, 2.14.0, 2.14.1, 2.14.2 -- 7 package versions matching "8.0.8 || 9.4.0 || 10.0.0 || 10.1.1 || 11.0.1 || >=12.0.0": 8.0.8, 9.4.0, 10.0.0, 10.1.1, 11.0.1, 12.0.0, 12.0.1 # opentelemetry-instrumentation-mongodb -- 4 package versions matching ">=6 <7": 6.0.0, 6.1.0, 6.2.0, 6.3.0 -- 14 package versions matching ">=5 <6": 5.0.0, 5.0.1, 5.1.0, 5.2.0, 5.3.0, 5.4.0, 5.5.0, 5.6.0, 5.7.0, 5.8.0, 5.8.1, 5.9.0, 5.9.1, 5.9.2 -- 32 package versions matching ">=4 <5": 4.0.0, 4.0.1, 4.1.0, 4.1.1, 4.1.2, 4.1.3, 4.1.4, 4.2.0, 4.2.1, 4.2.2, 4.3.0, 4.3.1, 4.4.0, 4.4.1, 4.5.0, 4.6.0, 4.7.0, 4.8.0, 4.8.1, 4.9.0, 4.9.1, 4.10.0, 4.11.0, 4.12.0, 4.12.1, 4.13.0, 4.14.0, 4.15.0, 4.16.0, 4.17.0, 4.17.1, 4.17.2 -- 38 package versions matching ">=3.3 <4": 3.3.0, 3.3.1, 3.3.2, 3.3.3, 3.3.4, 3.3.5, 3.4.0, 3.4.1, 3.5.0, 3.5.1, 3.5.2, 3.5.3, 3.5.4, 3.5.5, 3.5.6, 3.5.7, 3.5.8, 3.5.9, 3.5.10, 3.6.0, 3.6.1, 3.5.11, 3.6.2, 3.6.3, 3.6.4, 3.6.5, 3.6.6, 3.6.7, 3.6.8, 3.6.9, 3.6.10, 3.6.11, 3.6.12, 3.7.0, 3.7.1, 3.7.2, 3.7.3, 3.7.4 # opentelemetry-instrumentation-mysql2 -- 5 package versions matching "1.4.2 || 1.5.3 || 1.6.4 || 1.6.5 || 1.7.0": 1.4.2, 1.5.3, 1.6.4, 1.6.5, 1.7.0 -- 5 package versions matching "<3 >=2.3.2 || 2.3.0 || 2.2.5 || 2.1.0": 2.1.0, 2.2.5, 2.3.0, 2.3.2, 2.3.3 -- 28 package versions matching "<4 >=3.2.0 || 3.1.0 || 3.0.0": 3.0.0, 3.1.0, 3.2.0, 3.2.1, 3.2.2, 3.2.3, 3.2.4, 3.3.0, 3.3.1, 3.3.2, 3.3.3, 3.3.4, 3.3.5, 3.4.0, 3.4.1, 3.4.2, 3.4.3, 3.4.4, 3.4.5, 3.5.0, 3.5.1, 3.5.2, 3.6.0, 3.6.1, 3.6.2, 3.6.3, 3.6.4, 3.6.5 # opentelemetry-instrumentation-nestjs-core -- 1 package versions matching "6.11.11": 6.11.11 -- 5 package versions matching "^7.6.17 || 7.6.15 || 7.6.13 || 7.6.12": 7.6.12, 7.6.13, 7.6.15, 7.6.17, 7.6.18 -- 22 package versions matching "^8.2.0 || 8.1.2 || 8.1.1 || 8.0.11 || 8.0.9 || 8.0.6": 8.0.6, 8.0.9, 8.0.11, 8.1.1, 8.1.2, 8.2.0, 8.2.1, 8.2.2, 8.2.3, 8.2.4, 8.2.5, 8.2.6, 8.3.0, 8.3.1, 8.4.0, 8.4.1, 8.4.2, 8.4.3, 8.4.4, 8.4.5, 8.4.6, 8.4.7 # opentelemetry-instrumentation-pg -- 3 package versions matching "8.5.1 || 8.6.0 || 8.7.1": 8.5.1, 8.6.0, 8.7.1 # opentelemetry-instrumentation-pino -- 13 package versions matching "^8.16.2 || 8.12.1 || 8.8.0 || 8.4.0 || 8.0.0 || ^7.11.0 || 7.8.0 || 7.2.0 || ^6.13.3 || 5.17.0 || 5.14.0": 5.14.0, 5.17.0, 6.13.3, 7.2.0, 6.13.4, 7.8.0, 6.14.0, 7.11.0, 8.0.0, 8.4.0, 8.8.0, 8.12.1, 8.16.2 # opentelemetry-instrumentation-redis-4 -- 27 package versions matching "^4.0.0": 4.0.0, 4.0.1, 4.0.2, 4.0.3, 4.0.4, 4.0.5, 4.0.6, 4.1.0, 4.1.1, 4.2.0, 4.3.0, 4.3.1, 4.4.0, 4.5.0, 4.5.1, 4.6.0, 4.6.1, 4.6.2, 4.6.3, 4.6.4, 4.6.5, 4.6.6, 4.6.7, 4.6.8, 4.6.9, 4.6.10, 4.6.11 # opentelemetry-instrumentation-redis -- 15 package versions matching "^2.6.0 || ^3.0.0": 2.6.0, 2.6.1, 2.6.2, 2.6.3, 2.6.4, 2.6.5, 2.7.0, 2.7.1, 2.8.0, 3.0.0, 3.0.1, 3.0.2, 3.1.0, 3.1.1, 3.1.2 # opentelemetry-instrumentation-restify -- 2 package versions matching "^10.0.0 || ^11.1.0": 10.0.0, 11.1.0 # opentelemetry-instrumentation-winston -- 5 package versions matching "3.3.3 || 3.2.1 || 2.4.5 || 2.4.4 || 2.1.1": 2.1.1, 2.4.4, 3.2.1, 2.4.5, 3.3.3 ```
trentm commented 10 months ago

More details on that instrumentations' tests failed in the run linked above:

The fastify and pino failures were in the ESM tests. The setup-node step showed that v18.19.0 was installed. My theory is that these failures were due to https://github.com/open-telemetry/opentelemetry-js/issues/4356. If so, then those will go away when we get a new release of @opentelemetry/instrumentation.

The redis-4 test is two different failures. For Node 14:

2023-12-11T23:48:27.2486779Z -- required packages ["redis@4.6.9"]
2023-12-11T23:48:27.2487241Z -- installing ["redis@4.6.9"]
2023-12-11T23:48:33.4522889Z -- running test "npm run test" with redis (env: {})
2023-12-11T23:48:33.8119762Z
2023-12-11T23:48:33.8120759Z > @opentelemetry/instrumentation-redis-4@0.35.5 test
2023-12-11T23:48:33.8122256Z > nyc ts-mocha -p tsconfig.json --require '@opentelemetry/contrib-test-utils' 'test/redis.test.ts'
2023-12-11T23:48:33.8123149Z
2023-12-11T23:48:34.9531864Z
2023-12-11T23:48:34.9534054Z /home/runner/work/opentelemetry-js-contrib/opentelemetry-js-contrib/node_modules/@redis/client/dist/lib/client/index.js:195
2023-12-11T23:48:34.9535966Z             this[name.toLowerCase()] ??= this[name];
2023-12-11T23:48:34.9536749Z                                      ^^^
2023-12-11T23:48:34.9537215Z
2023-12-11T23:48:34.9537623Z SyntaxError: Unexpected token '??='
2023-12-11T23:48:34.9538486Z     at wrapSafe (internal/modules/cjs/loader.js:1029:16)
2023-12-11T23:48:34.9539968Z     at Module._compile (internal/modules/cjs/loader.js:1078:27)
2023-12-11T23:48:34.9542214Z     at Module.replacementCompile (/home/runner/work/opentelemetry-js-contrib/opentelemetry-js-contrib/node_modules/append-transform/index.js:60:13)

For Node 18, see above.

I think the redis-4 failure with Node 14 is likely about that version of redis just not supporting Node v14 anymore. I'll dig.

trentm commented 10 months ago

After those changes ^^, there are still two failures in the TAV tests with Node v18. Search for "-- fatal" in https://github.com/open-telemetry/opentelemetry-js-contrib/actions/runs/7206847040/job/19632480763 The failures are the same ones with "pino" and "fastify" as shown in the description above. My theory is that this is because Node v18.19.0 is used, and the fix is https://github.com/open-telemetry/opentelemetry-js/pull/4367 which is in @opentelemetry/instrumentation@0.46.0 which was just released today. So when this repo is updated to the new 1.19.0/0.46.0 release, then I expect the TAV tests to pass.

trentm commented 10 months ago

and the fix is open-telemetry/opentelemetry-js#4367 which is in

Actually, I mean https://github.com/open-telemetry/opentelemetry-js/pull/4357 which was the update to import-in-the-middle@1.6.0 which added support for Node v18.19.0. There have been more udpates to import-in-the-middle that I hadn't yet seen.

trentm commented 10 months ago

Current status update (also updated in the description):

Those issues cover all the current TAV failures that I know about. They are better to follow up on. I don't think there is much more value in this issue, so I'll close it now.