nspec / NSpec

A battle hardened testing framework for C# that's heavily inspired by Mocha and RSpec.
http://nspec.org/
MIT License
260 stars 57 forks source link

xitAsync prints an error message #130

Closed raghur closed 8 years ago

raghur commented 8 years ago

I have a few async itAsync specs that I need to temporarily disable. Intellisense prompts for xitAsync

 xitAsync["a revoked gateway cannot send events to Azure"] = async () =>
            {
            // spec code
            };

the examples are not run - but at the end of the test failures are printed like so

**** FAILURES ****

nspec. S03 Gateway Specs. revoking and reregistering gateway. a revoked gateway cannot send events to Azure.
'xit' cannot be set to an async delegate, please use 'xitAsync' instead
amirrajan commented 8 years ago

Could you provide the surrounding code too? Is there any before are act still being executed?

amirrajan commented 8 years ago

/cc @BrainCrumbz

raghur commented 8 years ago

here's a complete class

[Tag("dummy")]
    class Dummy_Specs : nspecbase
    {

        public void describe_scenario()
        {
            it["should run"] = () =>
            {
                5.should_be(5);
            };

            it["takes longer"] = () =>
            {
                Thread.Sleep(1000);
                5.should_be(5);
            };

            xitAsync["a pending spec"] = async () =>
            {
                await Task.Delay(10);
                "Should not be printed".Info();
            };
        }
    }
BrainCrumbz commented 8 years ago

Will have a look at it in these next days. Thanks for the full class example, that will be the first thing to try!

BrainCrumbz commented 8 years ago

.. but from a quick look it seems the problem lies in Context.HandleSkipped(...) here, which considers only a sync example, not an async one.

Will have to also double-check NSpec tests as well, whether this test case is missing or what.

BrainCrumbz commented 8 years ago

Hi there. Trying with following spec class and everything seems fine ... which maybe strange after all, but anyway:

    [Tag("dummy")]
    class Dummy_Specs : nspec
    {
        public void describe_scenario()
        {
            it["should run"] = () =>
            {
                5.Should().Be(5);
            };

            it["takes longer"] = () =>
            {
                Thread.Sleep(1000);
                5.Should().Be(5);
            };

            xitAsync["a pending spec"] = async () =>
            {
                await Task.Delay(10);
                Console.WriteLine("Should not be printed");
            };
        }
    }

The differences are:

  1. asserting through FluentAssertions, not those helpers deriving from NUnit
  2. deriving directly from nspec. Not sure what that nspecbase does in your case
  3. logging to console through Console.WriteLine, instead of that Info extension method

How are you running your spec? I'm going through our VS adapter. Here are the results:

dummy-test-results

Anyway will have a look at test cases and the aforementioned line code.

raghur commented 8 years ago

I'm running the test from the command line runner. Pasted your class in and ran it with similar results Have you tried running with the NspecRunner?

D:\code\iotnext\IotNext.ApiClient\src>packages\nspec.1.0.7\tools\NSpecRunner.exe IotNext.ApiTests\bin\Debug\IotNext.ApiTests.dll --tag dummy2

Dummy2 Specs
  describe scenario
    should run (31ms)
    takes longer (1s)
    a pending spec - PENDING

**** FAILURES ****

nspec. Dummy2 Specs. describe scenario. a pending spec.
'xit' cannot be set to an async delegate, please use 'xitAsync' instead
BrainCrumbz commented 8 years ago

Thanks for feedback. The error is actually visible also in VS test runner when debugging, but otherwise it gets swalloved somehow. Currently looking at it.

BrainCrumbz commented 8 years ago

Please have a look at commits until this one, they should fix this.

amirrajan commented 8 years ago

I'll merge and deploy as soon as I get a thumbs up. The commit looks good.

raghur commented 8 years ago

@braincrumbz the link is broken

raghur commented 8 years ago

Never mind... Found the commit... Do you want me to pull this commit and build and test? Don't know enough of the code though to content on it

BrainCrumbz commented 8 years ago

Sorry about that, somehow the link had only the commit ID, not the whole URL. Now's fixed.

If you could, it'd be enough to try testing with NSpec dll built out of that commit, in place of the one you already have within NSpec package folder in your project. I think this is the kind of thumbs up @amirrajan was expecting.

raghur commented 8 years ago

Ok - I built it with that commit (actually latest master - and still see the issue)

**** FAILURES ****

nspec. Dummy Specs. describe scenario. a pending spec.
'xit' cannot be set to an async delegate, please use 'xitAsync' instead
BrainCrumbz commented 8 years ago

Uhm. Trying from console with aforementioned Dummy_Specs class:

Dummy Specs
  describe scenario
    should run (357ms)
    takes longer (1s)
    a pending spec - PENDING

3 Examples, 0 Failed, 1 Pending
raghur commented 8 years ago

Ah - my fault - so I compiled against Nspec.dll from the commit you listed but used old nspecrunner

..Duh..

BrainCrumbz commented 8 years ago

Well, yeah, the change actually is in NSpec, not NSpecRunner, but then if the old NSpecRunner was still using the original NSpec.dll, I guess you were missing the change.

raghur commented 8 years ago

Closing the ticket