New Relic Node.js agent code base. Developers are welcome to create pull requests here, please see our contributing guidelines. For New Relic technical support, please go to http://support.newrelic.com.
Running versioned tests output
```
amqplib
• bluebird/regressions.tap.js run 1 of 28): waiting
bunyan
cassandra-driver
cjs-in-esm
cls
• connect/error-intercept.tap.js run 1 of 20): waiting
director
elastic
esm-package
• express/app-use.tap.js run 1 of 170): installing
express-esm
• fastify/add-hook.tap.js run 1 of 130): installing
generic-pool
• grpc/client-unary.tap.js run 1 of 56): waiting
grpc-esm
• hapi/capture-params.tap.js run 1 of 70): waiting
ioredis
langchain
• mongodb/bulk.tap.js run 1 of 84): waiting
• mongodb-esm/bulk.tap.mjs run 1 of 70): waiting
• mysql/basic-pool.tap.js run 1 of 40): waiting
• mysql2/basic-pool.tap.js run 1 of 50): waiting
nestjs
• openai/chat-completions.tap.js run 1 of 30): waiting
• pg/force-native.tap.js run 1 of 66): waiting
• pg-esm/force-native.tap.mjs run 1 of 66): waiting
pino
prisma
redis
• restify-post-7/capture-params.tap.js run 1 of 70): waiting
• restify-pre-7/capture-params.tap.js run 1 of 42): waiting
undici
winston
• winston-esm/winston.tap.mjs run 1 of 80): waiting
```
Notice that there are 84 possible tests for mongodb/bulk.tapjs. That means there will be at least 84 of those blocks printed to the test run output for that one suite.
The versioned test runner currently supports a -p <mode> switch, where mode can be either simple or pretty and defaults to pretty (which is what our CI is printing). We could switch our CI runs to use the "simple" mode and it would greatly improve things, but in this mode the test runner still outputs a lot of extraneous text. In simple mode, it prints a . character in place of the above quoted pretty status table. While this would make things easier, I think we can do better.
We should be able to add a "quiet" mode that will not print any status, except for bubbling up tap errors for failed tests, until such time as the full versioned test suite has finished. At that point, the tool should print its normal output that looks like:
workato-integration[bot]
commented
6 months ago
Description
The versioned test runner currently outputs a lot of text, making it very difficult to review errors during CI runs. See https://github.com/newrelic/node-newrelic/actions/runs/8542101178/job/23404883851#step:6:1 for an example. Basically, it writes out a status report for every completed child test suite for every parent test suite that looks like:
Running versioned tests output
``` amqplib • bluebird/regressions.tap.js run 1 of 28): waiting bunyan cassandra-driver cjs-in-esm cls • connect/error-intercept.tap.js run 1 of 20): waiting director elastic esm-package • express/app-use.tap.js run 1 of 170): installing express-esm • fastify/add-hook.tap.js run 1 of 130): installing generic-pool • grpc/client-unary.tap.js run 1 of 56): waiting grpc-esm • hapi/capture-params.tap.js run 1 of 70): waiting ioredis langchain • mongodb/bulk.tap.js run 1 of 84): waiting • mongodb-esm/bulk.tap.mjs run 1 of 70): waiting • mysql/basic-pool.tap.js run 1 of 40): waiting • mysql2/basic-pool.tap.js run 1 of 50): waiting nestjs • openai/chat-completions.tap.js run 1 of 30): waiting • pg/force-native.tap.js run 1 of 66): waiting • pg-esm/force-native.tap.mjs run 1 of 66): waiting pino prisma redis • restify-post-7/capture-params.tap.js run 1 of 70): waiting • restify-pre-7/capture-params.tap.js run 1 of 42): waiting undici winston • winston-esm/winston.tap.mjs run 1 of 80): waiting ```Notice that there are 84 possible tests for
mongodb/bulk.tapjs. That means there will be at least 84 of those blocks printed to the test run output for that one suite.The versioned test runner currently supports a
-p <mode>switch, wheremodecan be eithersimpleorprettyand defaults topretty(which is what our CI is printing). We could switch our CI runs to use the "simple" mode and it would greatly improve things, but in this mode the test runner still outputs a lot of extraneous text. In simple mode, it prints a.character in place of the above quoted pretty status table. While this would make things easier, I think we can do better.We should be able to add a "quiet" mode that will not print any status, except for bubbling up
taperrors for failed tests, until such time as the full versioned test suite has finished. At that point, the tool should print its normal output that looks like:Final versioned tests output
``` ✖ cassandra-driver/query.tap.js run 1 of 2): failure ✓ generic-pool (2) 3s ✓ express (34) 1m ✓ grpc-esm (2) 5s ✓ grpc (16) 1m ✓ mysql2 (15) 1m ✓ pino (2) 6s ✓ cls (1) 2s ✓ cjs-in-esm (1) 3s ✓ director (1) 2s langchain ✓ esm-package (1) 1s ✓ openai (3) 19s ✓ winston-esm (1) 2s ✓ winston (1) 3s ✓ restify-post-7 (35) 1m ✓ prisma (1) 8s ✓ undici (2) 15s ✓ mongodb (35) 2m ✓ nestjs (3) 1m Versions executed Folder: amqplib * amqplib(1): 0.10.3 Folder: bluebird * bluebird(2): 2.11.0, 3.7.2 Folder: bunyan * bunyan(2): 1.8.15, 2.0.5 Folder: cassandra-driver * cassandra-driver(2): 3.6.0, 4.7.2 Folder: cjs-in-esm * express(1): 4.18.2 * swagger-ui-express(1): 5.0.0 * axios(1): 1.4.0 Folder: cls * continuation-local-storage(1): 3.2.1 * async-listener(1): 0.6.10 Folder: connect * connect(2): 2.30.2, 3.7.0 Folder: director * director(1): 1.2.8 * express(1): 4.16 Folder: elastic * @elastic/elasticsearch(3): 7.13.0, 7.17.13, 8.13.0 * @elastic/transport(1): 8.4.1 Folder: esm-package * parse-json(1): 6.0.2 Folder: express-esm * express(2): 4.18.2, 4.19.2 * express-enrouten(1): 1.1 * ejs(1): 2.5.9 Folder: express * express(2): 4.18.2, 4.19.2 * express-enrouten(1): 1.1 * ejs(1): 2.5.9 Folder: fastify * fastify(3): 2.15.3, 3.29.5, 4.26.2 * middie(1): 5.3.0 Folder: generic-pool * generic-pool(2): 3.9.0, 2.5.4 Folder: grpc-esm * @grpc/grpc-js(1): 1.10.6 Folder: grpc * @grpc/grpc-js(2): >=1.4.0 <1.8.0, 1.10.6 Folder: hapi * ejs(1): 2.5.5 * @hapi/hapi(2): 20.3.0, 21.3.8 * @hapi/vision(1): 7.0.3 Folder: ioredis * ioredis(3): 3.2.2, 4.28.5, 5.3.2 Folder: langchain(SKIPPED) Folder: mongodb-esm * mongodb(3): 2.2.36, 3.7.4, 4.17.2 Folder: mongodb * mongodb(5): 2.2.36, 3.7.4, 4.17.2, 5.9.2, 6.5.0 Folder: mysql * mysql(1): 2.18.1 * generic-pool(1): 2.4 Folder: mysql2 * mysql2(3): 1.7.0, 2.3.3, 3.9.3 * generic-pool(1): 2.4 <2.5 Folder: nestjs * @nestjs/cli(3): 8.2.8, 9.5.0, 10.3.2 Folder: openai * openai(1): 4.32.1 Folder: pg-esm * pg(2): >=8.2 <8.8, 8.11.5 * pg-native(2): 2.2.0, 3.0.1 Folder: pg * pg(2): >=8.2 <8.8, 8.11.5 * pg-native(2): 2.2.0, 3.0.1 Folder: pino * pino(2): 7.11.0, 8.19.0 * flush-write-stream(1): 2.0.0 * split2(1): 4.1.0 Folder: prisma * @prisma/client(1): 5.12.0 * prisma(1): 5.12.0 Folder: redis * redis(6): >=2.0.0 < 2.3.0, >=2.3.0 < 2.4.0, >=2.4.0 < 2.6.0, 2.8.0, 3.1.2, 4.6.13 Folder: restify-post-7 * restify(5): 7.7.0, 8.6.1, 9.1.0, 10.0.0, 11.1.0 * express(1): 4.16 * restify-errors(1): 6.1.1 Folder: restify-pre-7 * restify(2): 5.2.1, 6.4.0 * express(1): 4.16 * restify-errors(1): 6.1.1 Folder: undici * undici(2): 4.16.0, 5.28.4 Folder: winston-esm * winston(1): 3.13.0 * winston-transport(1): 4.7.0 Folder: winston * winston(1): 3.13.0 =============================================================== =============================================================== FAIL (3) packages: pg@>=8.2 <8.8, pg-native@2.2.0 file: /Users/jsumners/Projects/team-repos/node-newrelic/test/versioned/pg-esm/force-native.tap.mjs packages: pg@>=8.2 <8.8, pg-native@2.2.0 file: /Users/jsumners/Projects/team-repos/node-newrelic/test/versioned/pg/force-native.tap.js packages: cassandra-driver@3.6.0 file: /Users/jsumners/Projects/team-repos/node-newrelic/test/versioned/cassandra-driver/query.tap.js ```Acceptance Criteria