Code coverage seems to be collected incorrectly

sam-pastoriza commented 3 years ago

I am looking to upgrade several projects to Vue 3 and I am running into an interesting issue when collecting code coverage. For some reason, when switching to Vue 3, the code coverage statistics are incorrect, but the tests are running correctly. Below is a repository that minimally reproduces the issue I am running into. Also attached are a couple of screenshots that show the difference in coverage between Vue 2 and Vue 3.

https://github.com/sam-pastoriza/vue-3-coverage

With Vue 2, you get the following coverage in the repo. With Vue 3, you get the following coverage, without changing the tests.

A readme file in that repository details how I switch between Vue 2 and Vue 3.

A couple of notes on the structure of the project I am working with.

We use vue-class-component as a library. I don't think this is causing the issue, but I could be wrong.
We are required to split up the components into separate .ts and .vue files.

Many thanks for the help. Hopefully this is the right place to post this issue.

sam-pastoriza commented 3 years ago

Splitting the components up seems to cause the issue. It looks like the latest version of the package doesn't cover code that is include via a script. This only occurs when upgrading from Vue 2 -> Vue 3.

lmiller1990 commented 3 years ago

This is most likely a bug in vue-jest which isn't handling source maps correctly - test utils just provides some basic functions like mount, the problem lies in the mapping between the source (vue file) and output (compiled via vue-jest into something Jest can read).

Either way this is a very good reproduction, thank you!

I'll move this issue to vue-jest, which I desperately need to find time to work on...

jolting commented 3 years ago

Try setting the fileName option here. https://github.com/vuejs/vue-jest/blob/acab46325633e641f245bcce4d12ed5259d1ea96/lib/process.js#L107

Should be filename or template.src if set.

sam-pastoriza commented 3 years ago

@lmiller1990 Thanks for the help! Much appreciated.

sam-pastoriza commented 3 years ago

@jolting Is that something I can do on my end in the tsconfig.json?

malhotrasaksham commented 3 years ago

@jolting Is that something I can do on my end in the tsconfig.json?

I guess not. He is referring to the file vue-jest/lib/process.js

jolting commented 3 years ago

@saksham-malhotra yeah, t's a code change. I ran into similar things when the file name was wrong on the source maps. module.tsx is the default for transpileModule if you don't give it a filename. See here https://github.com/microsoft/TypeScript/blob/2428ade1a91248e847f3e1561e31a9426650efee/src/services/transpile.ts#L50

jolting commented 3 years ago

@saksham-malhotra try #330

sam-pastoriza commented 3 years ago

I can try it when I get a chance today

lmiller1990 commented 3 years ago

It is actually REALLY easy to test if this work locally for this package. There is no babel, not typescript, no compilation, no bullshit, all you need to do is modify the code in node_modules/vue-jest/lib and run jest with --no-cache and you'll see if your fix worked. This is how I normally work on vue-jest for problems that are difficult to reproduce.

If you find a fix, let's work together and get it merged. Getting everything to work is pretty tricky due to the amount of transformations that are applied, eg often you go from vue -> ts -> js etc... but it would be great to get code coverage working accurately.

jolting commented 3 years ago

@lmiller1990 #331 should fix this.

lmiller1990 commented 3 years ago

🍻 Merged. thanks!

I usually do release at the end of the week, please wait a few days everyone.

sam-pastoriza commented 3 years ago

Following up on this, it looks like it does work better. It still doesn't exactly match the output of Vue 2, but this works for me in my case.

jolting commented 3 years ago

The vue2 output was wrong.

sam-pastoriza commented 3 years ago

@lmiller1990 Any chance of this update being released soon?

lmiller1990 commented 3 years ago

Hi! Sure. I will do a release by the end of this week, since we had a bunch of stuff merged up. I didn't realize it had been so long since we did a release on the next branch.

We might want to look into sem-release so we can have stuff in the wild asap at some point 🤔

lmiller1990 commented 3 years ago

It's out:

https://github.com/vuejs/vue-jest/releases/tag/v5.0.0-alpha.9

vuejs / vue-jest

Code coverage seems to be collected incorrectly #329