search

microsoft / navcontainerhelper

Official Microsoft repository for BcContainerHelper, a PowerShell module, which makes it easier to work with Business Central Containers on Docker.
MIT License
389 stars 248 forks source link

Get-TestsFromBCContainer often needs a second try #3589

Open mishof opened 5 months ago

mishof commented 5 months ago

When we use Get-TestsFromBCContainer, the first run often returns empty. The second run, with the same parameters, returns the Tests from the BC container

script: $iteration = 1 while ($iteration -le 3) { Write-Host ('Try {0}.' -f $iteration) $tests = Get-TestsFromBCContainer -containerName $PipeLineName -credential $credential -extensionId $_.AppId -ignoreGroups
if ($tests) { break } $iteration++ }

output: Try 1. Setting test codeunit range '' Try 2. Setting test codeunit range '' Connecting to http://localhost:80/BC/cs?tenant=default&company=CRONUS%20International%20Ltd. Setting test codeunit range '' Codeunit 72008 .

... running all tests after this as expected.

It happens when the container is newly created and the test app is installed after this. On a container that is running for a while with test app in it for a while, we do seem to get results at "Try 1". But it could be Try 1 has been a previous run long before that. Not sure if it ever worked at once all the time.

BcContainerHelper version 6.0.19 BcContainerHelper is running as administrator Host is Microsoft Windows Server 2022 Standard - 10.0.20348.2340 Docker Client Version is 25.0.4 Docker Server Version is 25.0.4

freddydk commented 4 months ago

Probably a timing issue (as you hint to yourself) If you can provide an end 2 end script, which repros the problem, I can try this when back from vacation.

mishof commented 4 months ago

@freddydk I did some more testing. When I use a standard testapp 'Microsoft_Tests-Bank.app' (after installing our products in the same script-run) in stead of our own I get results on Try 1. But placing our own testapp in exactly that position of events, I get results on Try 2. Maybe it is the number of tests, or the kind of tests, in our test app that require more time for initiation. I didn't create the tests, so I'll leave it at this. It isn't that big of an issue (once figured out a second run is needed sometimes 🙂).

Enjoy your vacation!

New-BcContainer -accept_eula -containerName $containerName -credential $credential -auth $auth -imageName 'bc24-test:latest' -memoryLimit 8G -updateHosts -shortcuts None -enableTaskScheduler:$false -includeTestToolkit ` -includeTestLibrariesOnly

$TestApp = 'C:\NAVDVD\Applications\BaseApp\Test\Microsoft_Tests-Bank.app' Invoke-ScriptInBcContainer -containerName $containerName -useSession:$false -ScriptBlock { Param($from, $to) Copy-Item -Path $from -Destination $to -force } -ArgumentList $TestApp, 'c:\run\my'

$RunMyPath = ((Get-BcContainerSharedFolders -containerName $containerName).GetEnumerator() | Where-Object { $_.Value -eq "c:\run\my" }).Key $NavAppInfo = Get-BcContainerAppInfo -containername $containerName -appFilePath ('{0}\Microsoft_Tests-Bank.app' -f $RunMyPath)

Publish-BcContainerApp -containername $containerName -appFile ('{0}\Microsoft_Tests-Bank.app' -f $RunMyPath) -skipVerification -sync ` -install

$iteration = 1 while ($iteration -le 3) { Write-Host ('Try {0}.' -f $iteration) $tests = Get-TestsFromBCContainer -containerName $containerName -credential $credential -extensionId $NavAppInfo.AppId -ignoreGroups
if ($tests) { break } $iteration++ }

$tests

... Try 1. Setting test codeunit range ''

Tests

{GetBankAccountNoReturnsIBANWhenIsNotEmpty_BankAccount, GetBankAccountNoReturnsBankaccountNoWhenIBANIsEmpty_BankAccount, GetBankAccountNoThrowsErrorIf... {TransferToGenJnlLineWithBalAcc, TransferToGenJnlLineWithBalAccAndPostWithoutGettingApplyUpdated, TransferToGenJnlAndPostPlusGetApplyUpdated, Transfer... {CRUDBankAccount, CRUDSWIFT, CheckBankAccIBANConfirmYes, CheckBankAccIBANConfirmNo...}
{OneBankEntryMoreRecLinesFullMatch, OneBankEntryMoreRecLinesMatchDetails, OneBankEntryMoreRecLinesAmtVsDesc, OneBankEntryMoreRecLinesAmtVsDocNo...}
{AddToMatchPartialBRL, AddToMatchPartialBLE, ReplaceMatch, RemoveManyToMany...}
{MatchDirectDebitCollectionForCAMT, MatchDirectDebitCollectWhenVendLedgerEntryWithTransactionIDExists, MatchDirectDebitCollectWhenCollectionStatusCanc... {TestInsertingLineUpdatesTheScore, TestRenamingLineUpdatesTheScore, TestInsertingLinesWithSameConfidenceDifferentPriorities, TestSettingPrioritySmalle... {TestNoMatchFoundCustomerEntries, TestNoMatchRulesAreIncludedInProposals, TestCustNoMatchOutsideThreshold, TestCustSingleAmountMatch...}
{SalesAppliedPmtEntryWithPmtToleranceWhenConfirmPmtDiscTolWarningOnAutomaticApplication, SalesAppliedPmtEntryWithNoPmtToleranceWhenDoNotConfirmPmtDisc... {TestPmtWithDim, TestPmtWithDim2, TestPmtWithDim3, TestPmtWithBlockedDim...}
{AutomaticApplyToEntry, RejectApplication, RejectApplicationFromPaymentJournal, UnapplyApplication...}
{TestApplied_OnCrMemo, TestApplied_OnCrMemo_WithInv, TestApplied_OnCrMemo_WithInvAndCrMemo, TestApplied_ToInv_WheneTherIsNothingToApplyLeft...}
{TestApplyWhenAmountIsMissing, TestApplyWithExcessAmount, TestApplyOnMultipleRecords, TestUncheckingApplySingleRow...}
{CustomerPaidDiscountedAmountOnTime, CustomerPaidDiscountedAmountAfterDiscountDate, CustomerPaidFullAmountInsteadOfDiscountedAmount, CustomerPaidFullA... {TestXSalesOnePmtPeformance}
{UpdateBankAccount, ChangeCurrencyOnBankAccount, ChangeCurrencyOnBankAccountFromBlankToLCY, BankAccountPostingGroup...}
{ImportNewStatementToNonSetBankAcc, ImportNewStatement_NoBankAcc, ImportNewStatement_Only1BankAcc_WithoutFileFormat, ImportNewStatement_StmntNo...}
{t273_BankAccountNo_ValidateFor_TypeBankReconciliation, t273_BankAccountNo_ValidateFor_TypeBankReconciliation_LastStmntNoIsExists, t273BankAccountNo...

BB97GER commented 3 months ago

@freddydk I can confirm the issue reported by @mishof . I am using AL-Go with CI, and I can verify that when there is initialization code in the Test App, I do not receive any results from Get-TestsFromBCContainer. The output in the pipeline looks like this:

2024-08-21T09:41:04.0751238Z ##[group]Running tests
2024-08-21T09:41:04.0760228Z   _____                   _               _            _
2024-08-21T09:41:04.0761877Z  |  __ \                 (_)             | |          | |
2024-08-21T09:41:04.0762628Z  | |__) |   _ _ __  _ __  _ _ __   __ _  | |_ ___  ___| |_ ___
2024-08-21T09:41:04.0770047Z  |  _  / | | | '_ \| '_ \| | '_ \ / _` | | __/ _ \/ __| __/ __|
2024-08-21T09:41:04.0774401Z  | | \ \ |_| | | | | | | | | | | | (_| | | ||  __/\__ \ |_\__ \
2024-08-21T09:41:04.0775114Z  |_|  \_\__,_|_| |_|_| |_|_|_| |_|\__, |  \__\___||___/\__|___/
2024-08-21T09:41:04.0775744Z                                    __/ |
2024-08-21T09:41:04.0776177Z                                   |___/
2024-08-21T09:41:04.1273882Z Running tests for App 8bb7dda9-b08c-4ce4-88fd-42014d2dac87 in 
                                                   C:\actionsrunner\_work\LearnLabAL\LearnLabAL\LearnLabTestApp
2024-08-21T09:41:04.1823705Z Using Container
2024-08-21T09:41:17.8655643Z Connecting to http://localhost:80/BC/cs?tenant=default
2024-08-21T09:41:38.9608030Z Setting test codeunit range ''
2024-08-21T09:41:40.5694978Z 
2024-08-21T09:41:40.5695406Z Running tests took 36 seconds
2024-08-21T09:41:40.5706850Z Copying test results to output
2024-08-21T09:41:40.5759224Z ##[endgroup]

After I removed the initialization in my Test App (OnInstallAppPerCompany() trigger), it worked on the first attempt. The result:

2024-08-21T09:19:31.0748359Z ##[group]Running tests
2024-08-21T09:19:31.0763752Z   _____                   _               _            _
2024-08-21T09:19:31.0764435Z  |  __ \                 (_)             | |          | |
2024-08-21T09:19:31.0764931Z  | |__) |   _ _ __  _ __  _ _ __   __ _  | |_ ___  ___| |_ ___
2024-08-21T09:19:31.0765535Z  |  _  / | | | '_ \| '_ \| | '_ \ / _` | | __/ _ \/ __| __/ __|
2024-08-21T09:19:31.0765948Z  | | \ \ |_| | | | | | | | | | | | (_| | | ||  __/\__ \ |_\__ \
2024-08-21T09:19:31.0766329Z  |_|  \_\__,_|_| |_|_| |_|_|_| |_|\__, |  \__\___||___/\__|___/
2024-08-21T09:19:31.0766652Z                                    __/ |
2024-08-21T09:19:31.0766932Z                                   |___/
2024-08-21T09:19:31.1187319Z Running tests for App 8bb7dda9-b08c-4ce4-88fd-42014d2dac87 in C:\actions- 
                                                   runner\_work\LearnLabAL\LearnLabAL\LearnLabTestApp
2024-08-21T09:19:31.1604363Z Using Container
2024-08-21T09:19:38.7160469Z Connecting to http://localhost:80/BC/cs?tenant=default
2024-08-21T09:20:44.2120104Z Setting test codeunit range ''
2024-08-21T09:20:50.0574908Z   Codeunit 60100 Binary Search Test Success (0.192 seconds)
2024-08-21T09:20:50.0716460Z     Testfunction BinarySearchElementFound Success (0.183 seconds)
2024-08-21T09:20:50.0749219Z     Testfunction BinarySearchElementNotFound Success (0.003 seconds)
2024-08-21T09:20:50.0762793Z     Testfunction BinarySearchEmptyList Success (0.003 seconds)
2024-08-21T09:20:50.0783494Z     Testfunction BinarySearchSingleElementFound Success (0 seconds)
2024-08-21T09:20:50.0791113Z     Testfunction BinarySearchSingleElementNotFound Success (0.003 seconds)
2024-08-21T09:20:52.3443366Z   Codeunit 60102 Linear Search Test Success (0.023 seconds)
2024-08-21T09:20:52.3455775Z     Testfunction LinearSearchElementFound Success (0.01 seconds)
2024-08-21T09:20:52.3474212Z     Testfunction LinearSearchElementNotFound Success (0.003 seconds)
2024-08-21T09:20:52.3485370Z     Testfunction LinearSearchEmptyList Success (0.004 seconds)
2024-08-21T09:20:52.3495195Z     Testfunction LinearSearchSingleElementFound Success (0.003 seconds)
2024-08-21T09:20:52.3505114Z     Testfunction LinearSearchSingleElementNotFound Success (0.003 seconds)
2024-08-21T09:20:55.2480032Z   Codeunit 60103 Bubble Sort Test Success (0.034 seconds)
2024-08-21T09:20:55.2504033Z     Testfunction BubbleSort_SimpleUnsortedList Success (0.017 seconds)
2024-08-21T09:20:55.2515951Z     Testfunction BubbleSort_AlreadySortedList Success (0.004 seconds)
2024-08-21T09:20:55.2531433Z     Testfunction BubbleSort_SingleElementList Success (0.004 seconds)
2024-08-21T09:20:55.2543744Z     Testfunction BubbleSort_EmptyList Success (0.003 seconds)
2024-08-21T09:20:55.2558504Z     Testfunction BubbleSort_ListWithDuplicates Success (0.006 seconds)
2024-08-21T09:20:57.7044512Z   Codeunit 60104 Insertion Sort Test Success (0.024 seconds)
2024-08-21T09:20:57.7059235Z     Testfunction InsertionSort_SimpleUnsortedList Success (0.01 seconds)
2024-08-21T09:20:57.7068342Z     Testfunction InsertionSort_AlreadySortedList Success (0.003 seconds)
2024-08-21T09:20:57.7075555Z     Testfunction InsertionSort_SingleElementList Success (0.004 seconds)
2024-08-21T09:20:57.7084557Z     Testfunction InsertionSort_EmptyList Success (0.004 seconds)
2024-08-21T09:20:57.7094374Z     Testfunction InsertionSort_ListWithDuplicates Success (0.003 seconds)
2024-08-21T09:20:59.9128880Z   Codeunit 60105 Selection Sort Test Success (0.029 seconds)
2024-08-21T09:20:59.9144867Z     Testfunction SelectionSort_SimpleUnsortedList Success (0.013 seconds)
2024-08-21T09:20:59.9164477Z     Testfunction SelectionSort_AlreadySortedList Success (0.006 seconds)
2024-08-21T09:20:59.9175295Z     Testfunction SelectionSort_SingleElementList Success (0.003 seconds)
2024-08-21T09:20:59.9184602Z     Testfunction SelectionSort_EmptyList Success (0.004 seconds)
2024-08-21T09:20:59.9195195Z     Testfunction SelectionSort_ListWithDuplicates Success (0.003 seconds)
2024-08-21T09:21:01.5232535Z 
2024-08-21T09:21:01.5233200Z Running tests took 90 seconds
2024-08-21T09:21:01.5248057Z Copying test results to output
2024-08-21T09:21:01.5408763Z ##[endgroup]

I am using BcContainerHelper version 6.0.20.