icy-arctic-fox / spectator

Feature-rich testing framework for Crystal inspired by RSpec.
https://gitlab.com/arctic-fox/spectator
MIT License
103 stars 5 forks source link

Properly document pending syntax #11

Closed postmodern closed 3 years ago

postmodern commented 4 years ago

Based on the example in the wiki, I cannot figure out how to use the pending syntax, but nothing seem to compile correctly.

Examples

  pending "not implemented yet" do
    it "should do something" do
      expect(1).to be(1)
    end
  end
  it "should do something" do
    pending "not implemented yet" do
      expect(1).to be(1)
    end
  end
  it "should do something" do
    pending "not implemented yet"
    expect(1).to be(1)
  end

Error

Code in spec/test_spec.cr:5:5

 5 | it "should do something" do
     ^
Called macro defined in lib/spectator/src/spectator/dsl/examples.cr:6:5

 6 | macro it(description = nil, _source_file = __FILE__, _source_line = __LINE__, &block)

Which expanded to:

 > 1 |       
 > 2 |         def __temp_1095
               ^
Error: can't define def inside def
icy-arctic-fox commented 4 years ago

The pending keyword works the same as it does in Crystal's Spec. Just replace it with pending to mark a single test as pending.

context "foo" do
  it "bar" do
  end
end

becomes:

context "foo" do
  pending "bar" do
  end
end

Are you trying to mark an entire group (context) as pending? That isn't supported as of now, but is planned.

How is this for new documentation?


Individual test cases can be skipped by using the pending keyword in place of it. If preferred, skip and xit can also be used, which are used by RSpec.

require "./spec_helper"

Spectator.describe Nil do
  # Skip this test.
  # To run, change back to `it`.
  pending "this is skipped" do
    expect(true).to be_false
  end

  xit "so is this" do
    expect(true).to be_false
  end
end

The tests in the code above will not run. This can be useful to stub tests for features that are not complete.

Groups of tests can't be skipped yet. This is a planned future feature.

postmodern commented 4 years ago

I was trying to wrap a it block with a pending block, where the pending description explains why it's marked pending, thus preserving the it description and the expect logic. I ended up using xits to skip the various tests. https://github.com/postmodern/v4l2.cr/blob/master/spec/device_spec.cr

icy-arctic-fox commented 4 years ago

Ah, I see. That would be nice to have. I'll see what I can do.

icy-arctic-fox commented 4 years ago

I'm thinking of adding the following syntax to the next minor release:

pending "skipped for reasons" do
  it "does a thing" do
    # ...
  end

  context "something else" do
    it "does another thing" do
      # ...
    end
  end
end

Syntax from RSpec for pending and skipped example will be added as well: https://relishapp.com/rspec/rspec-core/v/3-9/docs/pending-and-skipped-examples/pending-examples https://relishapp.com/rspec/rspec-core/v/3-9/docs/pending-and-skipped-examples/skip-examples

Does this satisfy your use cases?

postmodern commented 4 years ago

FYI pending inside of it would also be acceptable, as it's identical to RSpec 3's syntax.

it "should do thing" do
  pending "still need to fix this"

  expect(subject.foo).to be == bar
end
icy-arctic-fox commented 3 years ago

House keeping :bellhop_bell: Please reopen if this issue hasn't been addressed or suits your needs.

Spectator v0.10 allows pending and skip inside of an example block. This matches RSpec. An x can be placed in front of a group (xcontext or xdescribe) and examples (xit) to skip the block. Also, a :pending or :skip tag can be added to groups and examples to do something similar.

This syntax (pending as a group) isn't possible and probably won't be made possible:

pending "group" do
  it "does a thing" do
    # ...
  end
end