search

vitest-dev / vitest

Next generation testing framework powered by Vite.
https://vitest.dev
MIT License
13.06k stars 1.17k forks source link

Assertion message lengths should be clamped #6306

Closed DMartens closed 3 months ago

DMartens commented 3 months ago

Describe the bug

Some assertion messages are too long, e.g. any assertion with an unhandled error which has a big object as its property which then gets serialized. Here the error should be shortened by giving each property a maximum string length.

The problem is the other way around for assertions with a .resolves modifier which assertion message only includes the error message (which may be shortened) . In this case it would be more helpful if the whole error gets serialized. There may be others but these two cases crop up regularly.

Reproduction

Example for messages getting cut off for .resolves:

async function mayThrow() {
    throw new Error(`Long message before the actual relevant data which may gets cut off: ${data}`);
}

it('test case', async () => {
    await expect(mayThrow()).resolves.toBe(true);
});

Example for a too big serialized error:

function mayThrow() {
    let error = new Error('message');
    error.data = document; // Any big enough object

    throw error;
}

it('test case', () => {
    expect(mayThrow()).toBe(true);
});

System Info

System:
    OS: Linux 6.10 Arch Linux
    CPU: (12) x64 AMD Ryzen 5 3600 6-Core Processor
    Memory: 2.96 GB / 15.54 GB
    Container: Yes
    Shell: 5.2.32 - /bin/bash
  Binaries:
    Node: 22.6.0 - /usr/bin/node
    Yarn: 1.22.22 - /usr/bin/yarn
    npm: 10.8.2 - /usr/bin/npm
    pnpm: 9.7.0 - /usr/bin/pnpm
    bun: 1.1.22 - /usr/bin/bun
    Watchman: 20240414.112832.0 - /usr/bin/watchman
  Browsers:
    Chromium: 127.0.6533.72
  npmPackages:
    @vitest/coverage-v8: 2.0.5 => 2.0.5 
    vitest: 2.0.5 => 2.0.5

Used Package Manager

pnpm

Validations

sheremet-va commented 3 months ago

Example for a too big serialized error:

Vitest prints error properties only in verbose reporter. If you don't like it, do not use verbose reporter.

DMartens commented 3 months ago

I am not using the verbose reporter but a custom reporter which uses this.ctx.logger.printError directly and cannot set "printProperties" (it is also not in its types). It seems it is not the same printError as defined here.

My concrete example is that eslint adds the current traversed AST node to the error which then get serialized:

Example ``` Serialized Error: { ruleId: 'rule-to-test/Defaults', currentNode: { type: 'CallExpression', start: ': Use node.range[0] instead of node.start', end: ': Use node.range[1] instead of node.end', loc: { start: { line: 2, column: 9, constructor: 'Function', offset: 'Function' }, end: { line: 49258, column: 11, constructor: 'Function', offset: 'Function' }, constructor: 'Function' }, range: [ 33, 1555852 ], callee: { type: 'FunctionExpression', start: ': Use node.range[0] instead of node.start', end: ': Use node.range[1] instead of node.end', loc: { start: { line: 2, column: 10, constructor: 'Function', offset: 'Function' }, end: { line: 86, column: 10, constructor: 'Function', offset: 'Function' }, constructor: 'Function' }, range: [ 34, 3545 ], id: null, expression: false, generator: false, async: false, params: [ { type: 'Identifier', start: ': Use node.range[0] instead of node.start', end: ': Use node.range[1] instead of node.end', loc: { start: { line: 2, column: 19, constructor: 'Function', offset: 'Function' }, end: { line: 2, column: 26, constructor: 'Function', offset: 'Function' }, constructor: 'Function' }, range: [ 43, 50 ], name: 'modules', parent: [Circular], constructor: 'Function' } ], body: { type: 'BlockStatement', start: ': Use node.range[0] instead of node.start', end: ': Use node.range[1] instead of node.end', loc: { start: { line: 2, column: 28, constructor: 'Function', offset: 'Function' }, end: { line: 86, column: 10, constructor: 'Function', offset: 'Function' }, constructor: 'Function' }, range: [ 52, 3545 ], body: [ { type: 'VariableDeclaration', start: ': Use node.range[0] instead of node.start', end: ': Use node.range[1] instead of node.end', loc: { start: { line: 4, column: 10, constructor: 'Function', offset: 'Function' }, end: { line: 4, column: 36, constructor: 'Function', offset: 'Function' }, constructor: 'Function' }, range: [ 114, 140 ], declarations: [ { type: 'VariableDeclarator', start: ': Use node.range[0] instead of node.start', end: ': Use node.range[1] instead of node.end', loc: { start: { line: 4, column: 14, constructor: 'Function', offset: 'Function' }, end: { line: 4, column: 35, constructor: 'Function', offset: 'Function' }, constructor: 'Function' }, range: [ 118, 139 ], id: { type: 'Identifier', start: ': Use node.range[0] instead of node.start', end: ': Use node.range[1] instead of node.end', loc: { start: { line: 4, column: 14, constructor: 'Function', offset: 'Function' }, end: { line: 4, column: 30, constructor: 'Function', offset: 'Function' }, constructor: 'Function' }, range: [ 118, 134 ], name: 'installedModules', parent: [Circular], constructor: 'Function' }, init: { type: 'ObjectExpression', start: ': Use node.range[0] instead of node.start', end: ': Use node.range[1] instead of node.end', loc: { start: { line: 4, column: 33, constructor: 'Function', offset: 'Function' }, end: { line: 4, column: 35, constructor: 'Function', offset: 'Function' }, constructor: 'Function' }, range: [ 137, 139 ], properties: [], parent: [Circular], constructor: 'Function' }, parent: [Circular], constructor: 'Function' } ], kind: 'var', parent: [Circular], constructor: 'Function' }, { type: 'FunctionDeclaration', start: ': Use node.range[0] instead of node.start', end: ': Use node.range[1] instead of node.end', loc: { start: { line: 7, column: 10, constructor: 'Function', offset: 'Function' }, end: { line: 28, column: 11, constructor: 'Function', offset: 'Function' }, constructor: 'Function' }, range: [ 194, 925 ], id: { type: 'Identifier', start: ': Use node.range[0] instead of node.start', end: ': Use node.range[1] instead of node.end', loc: { start: { line: 7, column: 19, constructor: 'Function', offset: 'Function' }, end: { line: 7, column: 38, constructor: 'Function', offset: 'Function' }, constructor: 'Function' }, range: [ 203, 222 ], name: '__webpack_require__', parent: [Circular], constructor: 'Function' }, expression: false, generator: false, async: false, params: [ { type: 'Identifier', start: ': Use node.range[0] instead of node.start', end: ': Use node.range[1] instead of node.end', loc: { start: { line: 7, column: 39, constructor: 'Function', offset: 'Function' }, end: { line: 7, column: 47, constructor: 'Function', offset: 'Function' }, constructor: 'Function' }, range: [ 223, 231 ], name: 'moduleId', parent: [Circular], constructor: 'Function' } ], body: { type: 'BlockStatement', start: ': Use node.range[0] instead of node.start', end: ': Use node.range[1] instead of node.end', loc: { start: { line: 7, column: 49, constructor: 'Function', offset: 'Function' }, end: { line: 28, column: 11, constructor: 'Function', offset: 'Function' }, constructor: 'Function' }, range: [ 233, 925 ], body: [ { type: 'IfStatement', start: ': Use node.range[0] instead of node.start', end: ': Use node.range[1] instead of node.end', loc: { start: { line: 10, column: 11, constructor: 'Function', offset: 'Function' }, end: { line: 12, column: 12, constructor: 'Function', offset: 'Function' }, constructor: 'Function' }, range: [ 297, 397 ], test: { type: 'MemberExpression', start: ': Use node.range[0] instead of node.start', end: ': Use node.range[1] instead of node.end', loc: { start: { line: 10, column: 14, constructor: 'Function', offset: 'Function' }, end: { line: 10, column: 40, constructor: 'Function', offset: 'Function' }, constructor: 'Function' }, range: [ 300, 326 ], object: { type: 'Identifier', start: ': Use node.range[0] instead of node.start', end: ': Use node.range[1] instead of node.end', loc: { start: { line: 10, column: 14, constructor: 'Function', offset: 'Function' }, end: { line: 10, column: 30, constructor: 'Function', offset: 'Function' }, constructor: 'Function' }, range: [ 300, 316 ], name: 'installedModules', parent: [Circular], constructor: 'Function' }, property: { type: 'Identifier', start: ': Use node.range[0] instead of node.start', end: ': Use node.range[1] instead of node.end', loc: { start: { line: 10, column: 31, constructor: 'Function', offset: 'Function' }, end: { line: 10, column: 39, constructor: 'Function', offset: 'Function' }, constructor: 'Function' }, range: [ 317, 325 ], name: 'moduleId', parent: [Circular], constructor: 'Function' }, computed: true, optional: false, parent: [Circular], constructor: 'Function' }, consequent: { type: 'BlockStatement', start: ': Use node.range[0] instead of node.start', end: ': Use node.range[1] instead of node.end', loc: { start: { line: 10, column: 42, constructor: 'Function', offset: 'Function' }, end: { line: 12, column: 12, constructor: 'Function', offset: 'Function' }, constructor: 'Function' }, range: [ 328, 397 ], body: [ { type: 'ReturnStatement', start: ': Use node.range[0] ```
sheremet-va commented 3 months ago

I am not using the verbose reporter but a custom reporter which uses this.ctx.logger.printError directly and cannot set "printProperties" (it is also not in its types)

You need to set verbose: false if you are using printError in the logger:

https://github.com/vitest-dev/vitest/blob/3c04fdc3cc87f0625072141db7e1fad44881f752/packages/vitest/src/node/logger.ts#L22