search

allure-framework / allure-js

Allure integrations for JavaScript test frameworks
https://allurereport.org/
Apache License 2.0
225 stars 120 forks source link

[allure-jest] Test start time is incorrect, which leads to incorrect test duration #1002

Closed maks-rafalko closed 1 month ago

maks-rafalko commented 3 months ago

Describe the bug

We are using v2.15.1 and this version incorrectly calculates tests' start time, which leads to incorrectly calculated Duration time on allure report UI.

The instance of AllureTest is created here on Jest's add_test event:

https://github.com/allure-framework/allure-js/blob/d426d79b63386c42c854ede724c36b582e9b3512/packages/allure-jest/src/AllureJest.ts#L95-L98

Then, we set time to test.testResult.start:

https://github.com/allure-framework/allure-js/blob/d426d79b63386c42c854ede724c36b582e9b3512/packages/allure-js-commons/src/current/AllureTest.ts#L21

The root of the bug is that we should save start time on test_start event, not on add_test. Here:

https://github.com/allure-framework/allure-js/blob/d426d79b63386c42c854ede724c36b582e9b3512/packages/allure-jest/src/AllureJest.ts#L101-L103

add_test event occures for all the tests cases of the fille at the same time, so this is simply not a correct event to save start time of the test..

To Reproduce

  1. Create a test file with N tests, let's say N=5
  2. in each test() case, add for example sleep for 3 seconds
  3. Result - duration of the test N1 will be 3 seconds, duration of the test N2 will be 6 seconds, of the test N3 9 seconds and so on

Expected behavior

  1. Create a test file with N tests, let's say N=5
  2. in each test() case, add for example sleep for 3 seconds
  3. Result - duration of the test N1 will be 3 seconds, duration of the test N2 will be 3 seconds, of the test N3 3 seconds and so on

Screenshots

Before:

image

After adding a fix:

image

The fix is pretty simple, we just need to save test.testResult.start time somewere here on test_start event:

https://github.com/allure-framework/allure-js/blob/d426d79b63386c42c854ede724c36b582e9b3512/packages/allure-jest/src/AllureJest.ts#L179

If you agree, I can try to provide a PR.

baev commented 3 months ago

Could you please re-check the issue using the latest 3.0.0-beta.3 release? All the integrations were revisited (& reworked) recently, we're about to release 3.0.0. If the issue still occurs, we'll be happy to fix it

baev commented 1 month ago

The issue should be fixed in the latest 3.0.0-beta releases.