search

rsm-hcd / AndcultureCode.Cli

and-cli command-line tool to manage the development of software applications
https://andculture.com
Apache License 2.0
14 stars 15 forks source link

dotnet-test command integration tests fail locally w/ .NET Core 3.1.x [Windows] #107

Closed brandongregoryscott closed 3 years ago

brandongregoryscott commented 4 years ago

Not sure what's going on here, this will require some digging. It fails running the whole suite, or just filtered down to and-cli-dotnet-test:

> and-cli@1.1.3 test C:\Users\bscot\acc\fork\AndcultureCode.Cli
> jest "and-cli-dotnet-test"

 FAIL  ./and-cli-dotnet-test.test.js (71.577s)
  and-cli-dotnet-test
    Run by project
      when no solution can be found
        √ it displays an error (661ms)
      when solution exists
        × given a passing test suite, it prints success message (17676ms)
        × given a solution with multiple projects, it runs each project individually (28565ms)
    Run by solution
      when no solution can be found
        √ it displays an error (647ms)
      when solution exists
        × given a passing test suite, it prints success message (17057ms)
    given '-h'
      √ it displays the help menu (612ms)
    given '--help'
      √ it displays the help menu (657ms)

  ● and-cli-dotnet-test › Run by project › when solution exists › given a passing test suite, it prints success message

    expect(received).toMatch(expected)

    Expected pattern: /Test Run Successful/g
    Received string:  "[and-cli] Recursively deleting 'bin' directories...
    [and-cli] No 'bin' directories found - skipping this clean step.
    [and-cli] Recursively deleting 'obj' directories...
    [and-cli] 'obj' directories deleted successfully!
    [and-cli] Running dotnet clean (via dotnet clean .) on the solution...
    Microsoft (R) Build Engine version 16.4.0+e901037fe for .NET Core·
    Copyright (C) Microsoft Corporation. All rights reserved.···
    Build started 9/14/2020 12:52:38 PM.·
         1>Project \"C:\\Users\\bscot\\acc\\fork\\AndcultureCode.Cli\\tmp-dotnet-test-ff45acf4-8399-423c-ae29-0466af70b795\\tmp-dotnet-test-ff45acf4-8399-423c-ae29-0466af70b795.sln\" on node 1 (Clean target(s)).·
         1>ValidateSolutionConfiguration:·
             Building solution configuration \"Debug|Any CPU\".·
         1>Project \"C:\\Users\\bscot\\acc\\fork\\AndcultureCode.Cli\\tmp-dotnet-test-ff45acf4-8399-423c-ae29-0466af70b795\\tmp-dotnet-test-ff45acf4-8399-423c-ae29-0466af70b795.sln\" (1) is building \"C:\\Users\\bscot\\acc\\fork\\AndcultureCode.Cli\\tmp-dotnet-test-ff45acf4-8399-423c-ae29-0466af70b795\\Project.Test\\Project.Test.csproj\" (2) on node 1 (Clean target(s)).·
         2>CoreClean:·
             Creating directory \"obj\\Debug\\netcoreapp3.1\\\".·
         2>Done Building Project \"C:\\Users\\bscot\\acc\\fork\\AndcultureCode.Cli\\tmp-dotnet-test-ff45acf4-8399-423c-ae29-0466af70b795\\Project.Test\\Project.Test.csproj\" (Clean target(s)).·
         1>Done Building Project \"C:\\Users\\bscot\\acc\\fork\\AndcultureCode.Cli\\tmp-dotnet-test-ff45acf4-8399-423c-ae29-0466af70b795\\tmp-dotnet-test-ff45acf4-8399-423c-ae29-0466af70b795.sln\" (Clean target(s)).···
    Build succeeded.·
        0 Warning(s)·
        0 Error(s)···
    Time Elapsed 00:00:00.36·
    [and-cli] Dotnet solution cleaned
    [and-cli] Restoring nuget packages (via dotnet restore tmp-dotnet-test-ff45acf4-8399-423c-ae29-0466af70b795.sln)...
    C:\\Program Files\\dotnet\\sdk\\3.1.102\\NuGet.targets(528,5): error : Value cannot be null. (Parameter 'path1') [C:\\Users\\bscot\\acc\\fork\\AndcultureCode.Cli\\tmp-dotnet-test-ff45acf4-8399-423c-ae29-0466af70b795\\Project.Test\\Project.Test.csproj]·
    [and-cli] [ERROR] Solution failed to restore. See output for details.
    "

      67 | 
      68 |                 // Assert
    > 69 |                 expect(result).toMatch(TEST_RUN_SUCCESSFUL_REGEXP);
         |                                ^
      70 |             });
      71 | 
      72 |             test("given a solution with multiple projects, it runs each project individually", async () => {

      at Object.test (and-cli-dotnet-test.test.js:69:32)

  ● and-cli-dotnet-test › Run by project › when solution exists › given a solution with multiple projects, it runs each project individually

    TypeError: Cannot read property 'length' of null

      83 | 
      84 |                 // Assert
    > 85 |                 expect(result.match(TEST_RUN_SUCCESSFUL_REGEXP)).toHaveLength(
         |                                                                  ^
      86 |                     2
      87 |                 );
      88 |             });

      at Object.test (and-cli-dotnet-test.test.js:85:66)

  ● and-cli-dotnet-test › Run by solution › when solution exists › given a passing test suite, it prints success message

    expect(received).toMatch(expected)

    Expected pattern: /Test Run Successful/g
    Received string:  "[and-cli] Recursively deleting 'bin' directories...
    [and-cli] No 'bin' directories found - skipping this clean step.
    [and-cli] Recursively deleting 'obj' directories...
    [and-cli] 'obj' directories deleted successfully!
    [and-cli] Running dotnet clean (via dotnet clean .) on the solution...
    Microsoft (R) Build Engine version 16.4.0+e901037fe for .NET Core·
    Copyright (C) Microsoft Corporation. All rights reserved.···
    Build started 9/14/2020 12:53:24 PM.·
         1>Project \"C:\\Users\\bscot\\acc\\fork\\AndcultureCode.Cli\\tmp-dotnet-test-9d32025c-a61b-436a-8c13-59ba0d9a7e71\\tmp-dotnet-test-9d32025c-a61b-436a-8c13-59ba0d9a7e71.sln\" on node 1 (Clean target(s)).·
         1>ValidateSolutionConfiguration:·
             Building solution configuration \"Debug|Any CPU\".·
         1>Project \"C:\\Users\\bscot\\acc\\fork\\AndcultureCode.Cli\\tmp-dotnet-test-9d32025c-a61b-436a-8c13-59ba0d9a7e71\\tmp-dotnet-test-9d32025c-a61b-436a-8c13-59ba0d9a7e71.sln\" (1) is building \"C:\\Users\\bscot\\acc\\fork\\AndcultureCode.Cli\\tmp-dotnet-test-9d32025c-a61b-436a-8c13-59ba0d9a7e71\\tmp-dotnet-test-9d32025c-a61b-436a-8c13-59ba0d9a7e71.csproj\" (2) on node 1 (Clean target(s)).·
         2>CoreClean:·
             Creating directory \"obj\\Debug\\netcoreapp3.1\\\".·
         2>Done Building Project \"C:\\Users\\bscot\\acc\\fork\\AndcultureCode.Cli\\tmp-dotnet-test-9d32025c-a61b-436a-8c13-59ba0d9a7e71\\tmp-dotnet-test-9d32025c-a61b-436a-8c13-59ba0d9a7e71.csproj\" (Clean target(s)).·
         1>Done Building Project \"C:\\Users\\bscot\\acc\\fork\\AndcultureCode.Cli\\tmp-dotnet-test-9d32025c-a61b-436a-8c13-59ba0d9a7e71\\tmp-dotnet-test-9d32025c-a61b-436a-8c13-59ba0d9a7e71.sln\" (Clean target(s)).···
    Build succeeded.·
        0 Warning(s)·
        0 Error(s)···
    Time Elapsed 00:00:00.43·
    [and-cli] Dotnet solution cleaned
    [and-cli] Restoring nuget packages (via dotnet restore tmp-dotnet-test-9d32025c-a61b-436a-8c13-59ba0d9a7e71.sln)...
    C:\\Program Files\\dotnet\\sdk\\3.1.102\\NuGet.targets(528,5): error : Value cannot be null. (Parameter 'path1') [C:\\Users\\bscot\\acc\\fork\\AndcultureCode.Cli\\tmp-dotnet-test-9d32025c-a61b-436a-8c13-59ba0d9a7e71\\tmp-dotnet-test-9d32025c-a61b-436a-8c13-59ba0d9a7e71.csproj]·
    [and-cli] [ERROR] Solution failed to restore. See output for details.
    "

      115 | 
      116 |                 // Assert
    > 117 |                 expect(result).toMatch(TEST_RUN_SUCCESSFUL_REGEXP);
          |                                ^
      118 |             });
      119 |         });
      120 |     });

      at Object.test (and-cli-dotnet-test.test.js:117:32)

----------------------------|---------|----------|---------|---------|-------------------------------
File                        | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
----------------------------|---------|----------|---------|---------|-------------------------------
All files                   |   46.19 |     19.9 |   33.88 |   46.74 | 
 modules                    |   30.26 |     1.79 |    5.88 |   30.65 | 
  commands.js               |     100 |      100 |     100 |     100 | 
  constants.js              |     100 |      100 |     100 |     100 | 
  dir.js                    |   41.67 |        0 |       0 |   41.67 | 21-35
  dotnet-build.js           |      40 |        0 |       0 |      40 | 22-60
  dotnet-clean.js           |   29.55 |        0 |       0 |   29.55 | 29-36,46-64,75-116
  dotnet-path.js            |   22.22 |     5.88 |   15.38 |   22.22 | 82-200,211,228-274
  dotnet-restore.js         |      40 |        0 |       0 |      40 | 20-51
  dotnet-test.js            |   22.12 |        0 |    6.25 |   22.12 | 71-95,108-127,138-148,154-312
  echo.js                   |      20 |        0 |       0 |   20.59 | 32-87
  file.js                   |      25 |        0 |       0 |      25 | 28-63
  formatters.js             |    62.5 |        0 |   14.29 |     100 | 33-34
  variables.js              |     100 |      100 |     100 |     100 | 
 tests                      |   70.73 |    36.73 |   60.53 |   71.31 | 
  shared-specs.js           |   94.44 |       60 |   91.67 |   94.44 | 63,123
  test-utils.js             |   60.92 |    30.77 |   46.15 |   61.63 | 48,99,113,182,226-351        
 utilities                  |   86.67 |       50 |   93.33 |   87.67 | 
  command-string-factory.js |      92 |    33.33 |     100 |   91.67 | 59-60
  option-string-factory.js  |      84 |    57.14 |   88.89 |   85.71 | 30,83,87,95-96,102-103       
----------------------------|---------|----------|---------|---------|-------------------------------
Test Suites: 1 failed, 1 total
Tests:       3 failed, 4 passed, 7 total
Snapshots:   0 total
Time:        71.972s, estimated 73s
Ran all test suites matching /and-cli-dotnet-test/i.
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! and-cli@1.1.3 test: `jest "and-cli-dotnet-test"`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the and-cli@1.1.3 test script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\bscot\AppData\Roaming\npm-cache\_logs\2020-09-14T16_53_27_309Z-debug.log

This is my current dotnet --info output:

.NET Core SDK (reflecting any global.json):
 Version:   3.1.102
 Commit:    573d158fea

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.18363
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\3.1.102\

Host (useful for support):
  Version: 3.1.2
  Commit:  916b5cba26

.NET Core SDKs installed:
  2.1.202 [C:\Program Files\dotnet\sdk]
  2.1.503 [C:\Program Files\dotnet\sdk]
  2.2.102 [C:\Program Files\dotnet\sdk]
  3.0.102 [C:\Program Files\dotnet\sdk]
  3.1.102 [C:\Program Files\dotnet\sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.0.2 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.2 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

To install additional .NET Core runtimes or SDKs:
  https://aka.ms/dotnet-download

The Travis script runs the build on .NET Core 2.2.1, so something must have changed between there. Investigate possible issues with the way the dotnet cli scaffolds solutions/projects.

brandongregoryscott commented 4 years ago

Update: Confirmed that this runs on my Macbook Pro just fine:

> and-cli@1.2.0 test /Users/Brandon/acc/fork/AndcultureCode.Cli
> jest "and-cli-dotnet-test"

 PASS  ./and-cli-dotnet-test.test.js (60.808s)
  and-cli-dotnet-test
    Run by project
      when no solution can be found
        ✓ it displays an error (364ms)
      when solution exists
        ✓ given a passing test suite, it prints success message (16895ms)
        ✓ given a solution with multiple projects, it runs each project individually (22734ms)
    Run by solution
      when no solution can be found
        ✓ it displays an error (330ms)
      when solution exists
        ✓ given a passing test suite, it prints success message (17937ms)
    given '-h'
      ✓ it displays the help menu (329ms)
    given '--help'
      ✓ it displays the help menu (326ms)

----------------------------|---------|----------|---------|---------|-------------------------------
File                        | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s             
----------------------------|---------|----------|---------|---------|-------------------------------
All files                   |   46.19 |     19.9 |   33.88 |   46.74 |                               
 modules                    |   30.26 |     1.79 |    5.88 |   30.65 |                               
  commands.js               |     100 |      100 |     100 |     100 |                               
  constants.js              |     100 |      100 |     100 |     100 |                               
  dir.js                    |   41.67 |        0 |       0 |   41.67 | 21-35                         
  dotnet-build.js           |      40 |        0 |       0 |      40 | 22-60                         
  dotnet-clean.js           |   29.55 |        0 |       0 |   29.55 | 29-36,46-64,75-116            
  dotnet-path.js            |   22.22 |     5.88 |   15.38 |   22.22 | 82-200,211,228-274            
  dotnet-restore.js         |      40 |        0 |       0 |      40 | 20-51                         
  dotnet-test.js            |   22.12 |        0 |    6.25 |   22.12 | 71-95,108-127,138-148,154-312 
  echo.js                   |      20 |        0 |       0 |   20.59 | 32-87                         
  file.js                   |      25 |        0 |       0 |      25 | 28-63                         
  formatters.js             |    62.5 |        0 |   14.29 |     100 | 33-34                         
  variables.js              |     100 |      100 |     100 |     100 |                               
 tests                      |   70.73 |    36.73 |   60.53 |   71.31 |                               
  shared-specs.js           |   94.44 |       60 |   91.67 |   94.44 | 63,123                        
  test-utils.js             |   60.92 |    30.77 |   46.15 |   61.63 | 48,99,113,182,226-351         
 utilities                  |   86.67 |       50 |   93.33 |   87.67 |                               
  command-string-factory.js |      92 |    33.33 |     100 |   91.67 | 59-60                         
  option-string-factory.js  |      84 |    57.14 |   88.89 |   85.71 | 30,83,87,95-96,102-103        
----------------------------|---------|----------|---------|---------|-------------------------------
Test Suites: 1 passed, 1 total
Tests:       7 passed, 7 total
Snapshots:   0 total
Time:        60.987s, estimated 75s
Ran all test suites matching /and-cli-dotnet-test/i.
.NET Core SDK (reflecting any global.json):
 Version:   3.1.102
 Commit:    573d158fea

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  10.15
 OS Platform: Darwin
 RID:         osx.10.15-x64
 Base Path:   /usr/local/share/dotnet/sdk/3.1.102/

Host (useful for support):
  Version: 3.1.2
  Commit:  916b5cba26

.NET Core SDKs installed:
  2.1.4 [/usr/local/share/dotnet/sdk]
  2.1.300 [/usr/local/share/dotnet/sdk]
  2.2.207 [/usr/local/share/dotnet/sdk]
  2.2.402 [/usr/local/share/dotnet/sdk]
  3.1.102 [/usr/local/share/dotnet/sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.7 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.8 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.7 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.8 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.2 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.0.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.7 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.8 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.2 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET Core runtimes or SDKs:
  https://aka.ms/dotnet-download
brandongregoryscott commented 3 years ago

Stumbled upon this issue this morning - might be related. Investigate the way that TestUtils sets the HOME env variable when spawning processes, and maybe try the NUGET_PACKAGES env variable: https://github.com/NuGet/Home/issues/2960

brandongregoryscott commented 3 years ago

Update: It looks like it is related to the stripped down environment variables that we are assigning spawned processes from TestUtils. Merging overrides into process.env seems to fix the issue. Look into adding a Windows job to validate builds for our most common use-case environment. Would be awesome to integration test for dotnet core 2.x as well as 3.x, but if not possible, let's prefer the newer version.

brandongregoryscott commented 3 years ago

This should be fixed after pulling in the latest code from upstream/main into your fork. Feel free to post here with additional info if you are still seeing issues after cleaning, rebuilding and running the test:integration script.