search

Seddryck / NBi

NBi is a testing framework (add-on to NUnit) for Business Intelligence and Data Access. The main goal of this framework is to let users create tests with a declarative approach based on an Xml syntax. By the means of NBi, you don't need to develop C# or Java code to specify your tests! Either, you don't need Visual Studio or Eclipse to compile your test suite. Just create an Xml file and let the framework interpret it and play your tests. The framework is designed as an add-on of NUnit but with the possibility to port it easily to other testing frameworks.
http://www.nbi.io
Apache License 2.0
106 stars 37 forks source link

RunOnce doesn't seem to work #663

Closed fdw closed 2 years ago

fdw commented 2 years ago

I'm trying to run several tests with a shared setup that only needs to run once, before all tests (as the tests don't change the data). According to the docs, I can write something like this:

 <group name="Groupname">
    <setup>
      <tasks run-once="true">
        <sql-run name="Testsetup.sql" path="."/>
      </tasks>
    </setup>
    <test name="Test 1">
...

While the setup is executed, it looks to me like the run-once is ignored - the setup is run once for every test!

To find out whether I used the tag wrong, I had a look at the actual implementation. I think I found the problem, but I'm not really sure, so take the following with a large grain of salt:

In TestSuite.cs, the check for RunOnce looks like this:

var skip = false;
if (command is IGroupCommand)
{
    var groupCommand = (command as IGroupCommand);
    if (groupCommand.RunOnce)
        skip = groupCommand.HasRun;
}

However, command is a IGroupCommandArgs, so it cannot be a IGroupCommand. Additionally, IGroupCommandArgs does not have a RunOnce attribute. Update: I can't even find another reference to IGroupCommand :(

Did I use run-once correctly? Is it working, or is my interpretation of the code correct?

Seddryck commented 2 years ago

You are probably right that run-once is suffering from a bug. I've not checked in-depth but was able to reproduce. I'll take a look but probably not this week.

Seddryck commented 2 years ago

It was not really a single bug but many of them pilling up. I added many unit tests around this run-once for setup but won't have time to properly check if it's working end-to-end. Your feedback will be welcome on version 1.23.0-beta0192.

fdw commented 2 years ago

Probably I'm doing something wrong, but I cannot get the latest beta to work. Starting with 1.23.0-beta0172, I only get the error that Dynamitey, Version=2.0.9.136 is missing - but I cannot find anything referencing it.

Seddryck commented 2 years ago

Ok, will check tonight. Potentially something wrong in the packaging.

Seddryck commented 2 years ago

Huuuum, I tried to download the framework and it's working fine on my side. Ok, I got it by adding a setup to my test. Dynamitey is required by ImpromptuInterface, required for running some stuffs in the code for setup/cleanup (that I'm currently removing on v2.0). I will check why it's not in the packaging.

Seddryck commented 2 years ago

I fixed the issue around Dynamitey in the version 1.0.23-beta.195 available here