Permutation tests

[dburriss]

@dustinmoris I have written some PowerShell permutation tests for another template project at work using Invoke-Build. Wondering if you would be interested in me adding something like this? I found adding the --UsePaket a bit tedious testing different permutations after each test. What the tests do are: loop through permutations and do...

1. create a temp directory
2. generate template with specific options
3. restore, build, tests
4. report on errors but delete test directory if no errors

Interested?

[dustinmoris]

This sounds awesome. I was thinking of something similar as well the other day but thought "oh that sounds like more work than I have time for now" :)

If you would provide a PR for that this would be fantastic. This is definitely the way to go.

[dburriss]

I have made some progress on the tests. To see the progress check this ps1 file: https://github.com/dburriss/giraffe-template/blob/feature/template-tests/Test-Template.build.ps1

Currently it has the testing. I am going to add the paket.lock as a task. Might want to rename it. Maybe we could replace the build.ps1 file with this task runner although that does increase complexity slightly as it has a dependency on Invoke-Build.

Testing a single permutation

Testing different permutations

Success

[dburriss]

invoke-build Update-AllPaketLock task added to https://github.com/dburriss/giraffe-template/blob/feature/template-tests/Test-Template.build.ps1

[dburriss]

@dustinmoris development is complete on the automation of tests and auto updating the paket.lock. Test-Template.build.ps1 is not really fitting when it contains the paket.lock updates. I could split but then get into a ps1 explosion because there is some shared code so that would go in a 3rd ps1... Thinking Template.build.ps1 the build part is a convention used by Invoke-Build.

[dburriss]

@dustinmoris Added some readme content but not sure it is in the correct place? https://github.com/dburriss/giraffe-template/tree/feature/template-tests#contributing

[dburriss]

@dustinmoris What do you want to do about this? https://github.com/dburriss/giraffe-template/tree/feature/template-tests#testing-template-generation

The tests run and test a few permutations...

#some defaults
@{File='Test-Template.build.ps1'; }
@{File='Test-Template.build.ps1'; IncludeTests=$true}
@{File='Test-Template.build.ps1'; UsePaket=$true}
@{File='Test-Template.build.ps1'; IncludeTests=$true; UsePaket=$true}
#Giraffe
@{File='Test-Template.build.ps1'; ViewEngine="Giraffe"; IncludeTests=$false; UsePaket=$false}
@{File='Test-Template.build.ps1'; ViewEngine="Giraffe"; IncludeTests=$true; UsePaket=$false}
@{File='Test-Template.build.ps1'; ViewEngine="Giraffe"; IncludeTests=$false; UsePaket=$true}
@{File='Test-Template.build.ps1'; ViewEngine="Giraffe"; IncludeTests=$true; UsePaket=$true}
#Razor
@{File='Test-Template.build.ps1'; ViewEngine="Razor"; IncludeTests=$false; UsePaket=$false}
@{File='Test-Template.build.ps1'; ViewEngine="Razor"; IncludeTests=$true; UsePaket=$false}
@{File='Test-Template.build.ps1'; ViewEngine="Razor"; IncludeTests=$false; UsePaket=$true}
@{File='Test-Template.build.ps1'; ViewEngine="Razor"; IncludeTests=$true; UsePaket=$true}
#DotLiquid
@{File='Test-Template.build.ps1'; ViewEngine="DotLiquid"; IncludeTests=$false; UsePaket=$false}
@{File='Test-Template.build.ps1'; ViewEngine="DotLiquid"; IncludeTests=$true; UsePaket=$false}
@{File='Test-Template.build.ps1'; ViewEngine="DotLiquid"; IncludeTests=$false; UsePaket=$true}
@{File='Test-Template.build.ps1'; ViewEngine="DotLiquid"; IncludeTests=$true; UsePaket=$true}

It requires the installation of that Invoke-Build if that is acceptable.

[dustinmoris]

Hi,

Thanks for the suggested work with Invoke-Build.

I have implemented a very simple version of the permutation tests for now, but I think it works actually really well.

At the end of the original build script, after the giraffe-tempalte.{version}.nupkg has been created I first run the dotnet new giraffe --list command to check if an older giraffe-template is already installed. If yes then the script will uninstall it by running dotnet new -u giraffe-template.

Afterwards the script will immediately install the newly build template again.

When installation is complete the script will proceed to create a new .temp folder and then run the dotnet new command for each individual permutation (simple loop).

When that has completed (might take a bit) then the script will iterate through all test project folders and simply execute the ./build.cmd on Windows or ./build.sh on Unix environments. This was the easiest way to a) test that the build scripts work and b) build all projects and run the unit tests with a single command.

At last the script will go through all Paket enabled projects and run the paket update Giraffe command to update Giraffe and copy the updated paket.lock file into the correct template folder.

This might not be the most fancy way of doing it, but it works fairly simple, has been working really nicely for me now when updating the giraffe-template to Giraffe 1.1.0 and it also didn't require any new dependencies.