Seddryck / NBi

NBi is a testing framework (add-on to NUnit) for Business Intelligence and Data Access. The main goal of this framework is to let users create tests with a declarative approach based on an Xml syntax. By the means of NBi, you don't need to develop C# or Java code to specify your tests! Either, you don't need Visual Studio or Eclipse to compile your test suite. Just create an Xml file and let the framework interpret it and play your tests. The framework is designed as an add-on of NUnit but with the possibility to port it easily to other testing frameworks.
http://www.nbi.io
Apache License 2.0
106 stars 37 forks source link

Project file and configuration file path logic #447

Open fefeslon opened 5 years ago

fefeslon commented 5 years ago

Hi Cédric,

I have a question regarding the path logic used in the project file and configuration file.

Current state: .nunit: configfile - user has to write the relative path assembly path - user has to write the relative path .config: testSuite - user has to write the relative path settings - uses the current directory (no relative path)

Example:

<NUnitProject>
  <Settings activeconfig="Default" processModel="Default" domainUsage="Default" />
  <Config
    name="Default"
    binpathtype="Auto" appbase="C:\BI_Test\"
    configfile="TEST\Project\Cube\Cube1\cube_comparison.config">
    <assembly path="NBi\Framework\NBi.NUnit.Runtime.dll" />
  </Config>
</NUnitProject>
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
 <configSections>
  <section name="nbi" type="NBi.NUnit.Runtime.NBiSection, NBi.NUnit.Runtime"/>
 </configSections>
 <nbi testSuite="TEST\Project\Cube\Cube1\cube_comparison.nbits"
      settings="cube_comparison_settings.nbiset">
 <failure-report-profile
        threshold-sample-items="500"
        max-sample-items="5"
        expected-set="Sample"
        actual-set="Sample"
        analysis-set="Full"
    />
 </nbi>
</configuration>

And finally the question is: Is there any specific logic behind the pathing there?

I would say that the most logical should be to have an option to make the assembly path absolute (it should help a lot if there is one central NBi repository and there is no fixed level for the test repositories) and make other paths relative.

In that case the appbase could be used as current directory and all the project files could be mapped easily.

This has never been an issue until I got the more robust directory structure with different number of levels.

Cheers,

M.

Seddryck commented 5 years ago

I tried to be logical ... but Roma (or NBi) hasn't been built in one day and paths have been added at different moment, so I'm not sure that it's 100% consistent and this kind of feedback is important for me, appreciated and welcome. Your mileage will vary, but personally, I don't want to use absolute paths. Main reason is that we (in the teams I'm working) install the directory with NBi and test-suites on different places on our computers ... absolute paths are not an option and everything is relative to a root that is also the root of the project.

I also have constraints, NUnit file is not under NBi project team's control. From the top of my head, NUnit file supports relative and absolute paths ... also for appbase. I don't think we should find an inconsistency at this level (NUnit file). I agree with you that on the settings file, it's not logical ... the test-suite and settings file don't match in terms of the root (one using the appbase, the other using the directory of the config file). There is definitively a room for improvement for this field.

Now, last but not least, the file defined in the test-suite (CSV or queries) are relative to the test-suite and not to the appbase. It sounds more logical to me, when writing the test-suite you don't care about the infrastructure.

In conclusion, I agree with you that settings' root should be matching with test-suite's root and I'll change that in 2.0 ... if you're seeing other improvements at that level, feel free to share and discuss them.

fefeslon commented 5 years ago

Thanks for the explanation.

In our team we use the SVN and decided to use the same path so the option to use absolute path for the assembly looks logical to us.

To be more specific we have a lot of smaller projects, each of them in its own folder (sometimes with different directory level).

So in case of manual work (when the new project is being created) it means:

Yes this can be scripted/or be done automatically with some workaround, but I was just thinking if it could be done kind of naturally in the NBi (or am I missing something already existing?).

I am OK with the solution you mentioned (above), on the other hand if we have had an option to choose between relative/absolute path for the assembly (which is static and absolute for us) then we could use the appbase="." and get rid of all the manual stuff mentioned above.

Cheers.

Seddryck commented 5 years ago

I get your point but unfortunately it's not possible at the moment and implementing that would require to modify NUnit itself … wont-fix. When I'll work on 2.0, I'll keep an eye on this.

About some automation around the config and nunit files, the only supported path is the VisualStudio integration where the config and nunit files are automatically created but the design is one-framework-by-test-suite … surely not what you're trying to achieve here.

If I was in your position, I'd create a PowerShell script automatically fixing or creating nunit and config files. That'd be my best advice.

fefeslon commented 5 years ago

OK, we will manage it then.

Thanks for your answer anyway!