microsoft / navcontainerhelper

Official Microsoft repository for BcContainerHelper, a PowerShell module, which makes it easier to work with Business Central Containers on Docker.
MIT License
385 stars 247 forks source link

Run-TestsInBcContainer without specifying extensionId does not work #2890

Closed eclipses closed 1 year ago

eclipses commented 1 year ago

I'm trying to run all Microsoft Tests together with my Custom Tests in a DevOps Pipeline using Run-AlPipeline.

I'm not passing appfolders but only one app in the testfolders. In the test app I have an Install codeunit that creates the Test Suite called "DEFAULT" with range of the objects '50000..'.

I overwritten RunTestsInBcContainer of Run-AlPipeline to remove from the parameters the extensionId.

Here the result

Running tests for App 96c1fc3a-3d7b-40a7-ace4-1e232d49bd05 in C:\1\_work\9\s\TestApp

Connecting to [http://localhost:80/BC/cs?tenant=default](http://localhost/BC/cs?tenant=default)
Run-Tests, open page 130455
Show form 0001fd97-0000-0001-0c00-0000836bd2d2
Title: Command Line Test Tool Edit - Command Line Test Tool - DEFAULT
Disabling Test CodeunitName:Method
Using repeater based test-runner
Offset: 0
Offset: 0
Index:  0, Offset: 0, Count:  2
Row - lineType = 1, run = Yes, CodeunitId = 0, codeunitName = '', name = ''
Found Row - index = 1, rowIndex = 0/2, lineType = 1, run = Yes, CodeunitId = 0, codeunitName = '', name = ''

My feeling is that Run-TestsInBcContainer without specifying extensionId stopped working. According to the example and documentation it should run all tests.

I've tested also with the preview version of BCContainerHelper but same result.

eclipses commented 1 year ago

At this point https://github.com/microsoft/navcontainerhelper/blob/09f2cfd6ef3bd0c925915e8ea169f34646e22a89/AppHandling/PsTestFunctions.ps1#L546 The default test page is 130455 then in th next line set the extensionId even if there is not extensionId Set-ExtensionId -ExtensionId $extensionId -Form $form -ClientContext $clientContext -debugMode:$debugMode

freddydk commented 1 year ago

Will have a look. I would recommend NOT using testsuites and special test runners - but of course they should still work.

At some point in time, we will be refactoring test runners and allow running tests through the Dev Endpoint from VS Code. This will probably not be using AL Test Runners and Test Suites.

freddydk commented 1 year ago

At this point

https://github.com/microsoft/navcontainerhelper/blob/09f2cfd6ef3bd0c925915e8ea169f34646e22a89/AppHandling/PsTestFunctions.ps1#L546

The default test page is 130455 then in th next line set the extensionId even if there is not extensionId Set-ExtensionId -ExtensionId $extensionId -Form $form -ClientContext $clientContext -debugMode:$debugMode

Correct, but the Set-ExtensionId function returns immediately if extensionId is empty

I know that ALOps still supports the old way of running tests, so it should work.

However, if you can create a very small test app with a single test function and send it my way (including how you run it), then I will see where the error is.

eclipses commented 1 year ago

Hi, I've attached a test app that includes a custom test plus an install codeunit that creates the testsuite with all tests, mine plus the the Microsoft Standard tests.

TestApp.zip

I tried to run in different ways:

  1. Customized Run-AlPipeline to owerwrite RunTestsInBcContainer and remove the extensionId
    $RunTestsInBcContainer = {
    Param([Hashtable]$parameters)
    $parameters.Remove('extensionId')    
    $parameters
    Run-TestsInBcContainer @parameters
    }
  2. Create the container, deploy the extension and then run the test from PowerShell the command Run-TestsInBcContainer -containerName $containerName -credential $credential -detailed -debug -debugMode

In both cases I'm getting the same result.

freddydk commented 1 year ago

It looks like you are using the CAL Test Runner - the Run-TestsInBcContainer will per default use the AL Test Runner. I think you can change the test page to use the CAL Test Runner - but you should really create the test suite in the AL Test Runner objects as the CAL Test Runner might one day disappear.

eclipses commented 1 year ago

Hi Freddy, I'm sorry but I don't understand from your message how you identified thta I'm using CAL Test Runner. It's not my intention to use functionalities that will disappear. Can you please add more details on where I should apply the chnages?

you should really create the test suite in the AL Test Runner objects Please clarify this sentence

freddydk commented 1 year ago

image CAL Test Suite and the Run-Tests will execute tests from the AL Test Suite (I think)

freddydk commented 1 year ago

It has been many years since I looked at this last time - I don't use test suites anymore - but I am pretty certain that I remember that there are two different sets.

eclipses commented 1 year ago

Thank you, I fixed the code and made it working.