jestjs / jest

Delightful JavaScript Testing.
https://jestjs.io
MIT License
44.05k stars 6.44k forks source link

Feature request: Get detailed metadata from expect matchers in reporters #4547

Open esamattis opened 6 years ago

esamattis commented 6 years ago

I'd like to generate HTML reports from jest-image-snapshot with the diff images.

But doing it from a customer Jest reporter does not seem to be possible because because custom matchers such as the toMatchImageSnapshot cannot pass extra metadata (path to the snapshot diff file) to the reporters.

I'd like to be able to return the filename here and read it somehow in a onTestResult() method of a reporter.

Ref https://github.com/americanexpress/jest-image-snapshot/issues/13

cpojer commented 6 years ago

I'm trying to figure out what the implications are for returning the file name here, and what the downsides are, but I cannot think clearly right now. I definitely think we should find a way to support this.

@aaronabramov and @thymikee do you guys have any thoughts about this?

thymikee commented 6 years ago

Hm, this is another key in the object, so as long as we keep all properties initialized (at least in core matchers), we should still have this code monomorphic and perf shouldn't degrade I think. And some metadata may really be useful to create more powerful matchers.

cpojer commented 6 years ago

I took a break and have some more clarity. My concern was mostly around what exposing a file name will mean: Does returning an absolute path cause issues with caching? Should it be relative to the rootDir? More importantly: will this enable people to store more metadata in different files, without appropriate hooks to clean them up when a snapshot/test is removed? I worry that this is opening a can of worms that allows people to do crazy things and that we may need to provide a fuller API to hook into Jest that gives access to the cleanup hook etc.

esamattis commented 6 years ago

Hmm, I'm bit confused. I meant just adding possibility pass any metadata from a (custom) matcher to the (custom) reporters. Why does it matter if it's filename or any other string/object?

anescobar1991 commented 6 years ago

@cpojer jest-snapshot returns a few extra properties but I am not sure that custom reporters have access to these.

I think the ability for any metadata to be returned by a matcher and for reporters to pick up these properties is what @epeli is asking for. From the looks of the code comment in jest-snapshot this may already be possible though?

Prior99 commented 6 years ago

@anescobar1991 did you ever figure out whether it's possible? I tried to access the data passed to actual and expected from the matcher but with no results.

SimenB commented 5 years ago

Related #7594

JCMais commented 1 year ago

I am trying to add custom metadata inside a test to be read by a custom reporter (like test category, severity, etc), is this issue related to this in any way?

What is the current status of this?

It is still open but was assigned to the v26 milestone