cypress-io / cypress-grep

Filter tests using substring
137 stars 19 forks source link

Runs before block as well although the tests don't run #28

Closed vkolgi closed 3 years ago

vkolgi commented 3 years ago

One specific issue is that the before block runs for tests which are not supposed to run.

describe('My First Test', () => {

  it('Does not do much! @smoke', () => {
    expect(true).to.equal(true)
  })

  it("Does not do much either!, but shouldn't run", () => {
    expect(true).to.equal(true)
  })

})

describe('Tests with before block', () => {

  before('You know, setting up fixtures', () => {
    cy.visit('http://google.com')
    cy.log('setting up fixtures')
  })

  it('Tests with the before block', () => {
    expect(true).to.equal(true)
  })

  it("Tests with the before block, again", () => {
    expect(true).to.equal(true)
  })

})

Strangely the commands of the before block doesn't show up in the command log. Here if I grep with @smoke the tests in second describe block won't run but the before block does. Is there any way to skip them as well ?

bahmutov commented 3 years ago

What happens if you set it to it.skip for all tests in the second block? Does it still run the before hook? It could be how mocha and cypress execute it

Sent from my iPhone

On May 7, 2021, at 09:37, Vinayak @.***> wrote:

 One specific issue is that the before block runs for tests which are not supposed to run.

describe('My First Test', () => {

it('Does not do much! @smoke', () => { expect(true).to.equal(true) })

it("Does not do much either!, but shouldn't run", () => { expect(true).to.equal(true) })

})

describe('Tests with before block', () => {

before('You know, setting up fixtures', () => { cy.visit('http://google.com') cy.log('setting up fixtures') })

it('Tests with the before block', () => { expect(true).to.equal(true) })

it("Tests with the before block, again", () => { expect(true).to.equal(true) })

})

Strangely the commands of the before block doesn't show up in the command log. Here if I grep with @smoke the tests in second describe block won't run but the before block does. Is there any way to skip them as well ?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

vkolgi commented 3 years ago

Not sure if there is a workaround for this, I want to take a stab at this. From the code it looks like it's harder to get some sort of look ahead, where skip before block if corresponding tests are not supposed to run.

vkolgi commented 3 years ago

@bahmutov I tried with the it.skip for all the tests in second block. It still runs the before hook. Hmm, It would be costly to run before blocks of all the tests when we run subset of test suite (e.g. smoke), especially when these fixtures are used to fire APIs to setup the tests.

bahmutov commented 3 years ago

So this is mocha problem. I think in this case one would need to explicitly skip the suite and use tags

Sent from my iPhone

On May 7, 2021, at 09:47, Vinayak @.***> wrote:

 @bahmutov I tried with the it.skip for all the tests in second block. It still runs the before hook.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

vkolgi commented 3 years ago

Yeah since this is consistent with mocha, shall I close the issue ?

bahmutov commented 3 years ago

Yeah since this plug-in cannot really control this

Sent from my iPhone

On May 7, 2021, at 10:15, Vinayak @.***> wrote:

 Yeah since this is consistent with mocha, shall I close the issue ?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

vkolgi commented 3 years ago

Closing the issue, since this is consistent with mocha and plugin cannot do much.