search

pact-foundation / pact-net

.NET version of Pact. Enables consumer driven contract testing, providing a mock service and DSL for the consumer project, and interaction playback and verification for the service provider project.
https://pact.io
MIT License
832 stars 230 forks source link

PACT tests do not run under Mono on Linux #455

Open drew-cooper opened 1 year ago

drew-cooper commented 1 year ago

We have a multi-targeted test project (net472 and netcoreapp3.1) containing PACT Consumer tests. The tests fail to run for the net472 on our Ubuntu-based build image. The image has dotnet SDK 3.1 and 6.0 and Mono 6.6.0 installed.

Minimal viable repro:

PactMultitargetTest.csproj:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFrameworks>net472;netcoreapp3.1</TargetFrameworks>
    <LangVersion>10.0</LangVersion>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.4.0" />
    <PackageReference Include="NUnit" Version="3.13.3" />
    <PackageReference Include="NUnit3TestAdapter" Version="3.17.0" />
    <PackageReference Include="PactNet" Version="4.5.0" />
  </ItemGroup>

</Project>

SimpleTest.cs:

using System.Net;
using System.Net.Http;
using System.Threading.Tasks;
using NUnit.Framework;
using PactNet;

namespace PactMultitargetTest;

public class SimpleTest
{
    [Test]
    public async Task TestConsumerPactExecution()
    {
        var pact = Pact.V3("Consumer", "Provider").WithHttpInteractions();

        pact.UponReceiving("A request").WithRequest(HttpMethod.Get, "/")
            .WillRespond().WithStatus(HttpStatusCode.OK);

        await pact.VerifyAsync(async ctx =>
            {
                using var http = new HttpClient();
                await http.GetAsync(ctx.MockServerUri);
            });
    }
}

This runs as expected on Windows:

> dotnet test
<snip>
Test run for C:\my-repo-dir\PactMultitargetTest\bin\Debug\net472\PactMultitargetTest.dll (.NETFramework,Version=v4.7.2)
Microsoft (R) Test Execution Command Line Tool Version 17.4.0 (x64)
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.

Passed!  - Failed:     0, Passed:     1, Skipped:     0, Total:     1, Duration: 696 ms - PactMultitargetTest.dll (net472)
Test run for C:\my-repo-dir\PactMultitargetTest\bin\Debug\netcoreapp3.1\PactMultitargetTest.dll (.NETCoreApp,Version=v3.1)
Microsoft (R) Test Execution Command Line Tool Version 17.4.0 (x64)
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.

Passed!  - Failed:     0, Passed:     1, Skipped:     0, Total:     1, Duration: 100 ms - PactMultitargetTest.dll (netcoreapp3.1)

On Ubuntu I get this:

$ dotnet test
  Determining projects to restore...
  All projects are up-to-date for restore.
  PactMultitargetTest -> /mnt/c/Users/andrew.cooper/repos/test/PactMultitargetTest/bin/Debug/net472/PactMultitargetTest.dll
  PactMultitargetTest -> /mnt/c/Users/andrew.cooper/repos/test/PactMultitargetTest/bin/Debug/netcoreapp3.1/PactMultitargetTest.dll
  PactMultitargetTest -> /mnt/c/Users/andrew.cooper/repos/test/PactMultitargetTest/bin/Debug/net6.0/PactMultitargetTest.dll
Test run for /mnt/c/Users/andrew.cooper/repos/test/PactMultitargetTest/bin/Debug/net472/PactMultitargetTest.dll (.NETFramework,Version=v4.7.2)
Microsoft (R) Test Execution Command Line Tool Version 17.5.0 (x64)
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
  Failed TestConsumerPactExecution [126 ms]
  Error Message:
   System.DllNotFoundException : pact_ffi assembly:<unknown assembly> type:<unknown type> member:(null)
  Stack Trace:
    at (wrapper managed-to-native) PactNet.Interop.NativeInterop.LogToBuffer(PactNet.Interop.LevelFilter)
  at PactNet.PactExtensions.InitialiseLogging (PactNet.PactLogLevel level) [0x00067] in <c3835e33e6c34624a3900fd28f0ae514>:0
  at PactNet.PactExtensions.WithHttpInteractions (PactNet.IPactV3 pact, System.Nullable`1[T] port, PactNet.Models.IPAddress host) [0x0000b] in <c3835e33e6c34624a3900fd28f0ae514>:0
  at PactMultitargetTest.SimpleTest.TestConsumerPactExecution () [0x00022] in <25f0af484314489ebafdc4e264ba94be>:0
  at NUnit.Framework.Internal.TaskAwaitAdapter+GenericAdapter`1[T].GetResult () [0x00008] in <c804c596ec8e46f396062449e02bacdd>:0
  at NUnit.Framework.Internal.AsyncToSyncAdapter.Await (System.Func`1[TResult] invoke) [0x0003d] in <c804c596ec8e46f396062449e02bacdd>:0
  at NUnit.Framework.Internal.Commands.TestMethodCommand.RunTestMethod (NUnit.Framework.Internal.TestExecutionContext context) [0x00031] in <c804c596ec8e46f396062449e02bacdd>:0
  at NUnit.Framework.Internal.Commands.TestMethodCommand.Execute (NUnit.Framework.Internal.TestExecutionContext context) [0x00001] in <c804c596ec8e46f396062449e02bacdd>:0
  at NUnit.Framework.Internal.Execution.SimpleWorkItem+<>c__DisplayClass4_0.<PerformWork>b__0 () [0x00011] in <c804c596ec8e46f396062449e02bacdd>:0

  at NUnit.Framework.Internal.ContextUtils+<>c__DisplayClass1_0`1[T].<DoIsolated>b__0 (System.Object _) [0x00000] in <c804c596ec8e46f396062449e02bacdd>:0
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00071] in <de882a77e7c14f8ba5d298093dde82b2>:0
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in <de882a77e7c14f8ba5d298093dde82b2>:0
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state) [0x0002b] in <de882a77e7c14f8ba5d298093dde82b2>:0
  at NUnit.Framework.Internal.ContextUtils.DoIsolated (System.Threading.ContextCallback callback, System.Object state) [0x00025] in <c804c596ec8e46f396062449e02bacdd>:0
  at NUnit.Framework.Internal.ContextUtils.DoIsolated[T] (System.Func`1[TResult] func) [0x0001a] in <c804c596ec8e46f396062449e02bacdd>:0
  at NUnit.Framework.Internal.Execution.SimpleWorkItem.PerformWork () [0x0001b] in <c804c596ec8e46f396062449e02bacdd>:0

Failed!  - Failed:     1, Passed:     0, Skipped:     0, Total:     1, Duration: 126 ms - PactMultitargetTest.dll (net472)
Test run for /mnt/c/Users/andrew.cooper/repos/test/PactMultitargetTest/bin/Debug/netcoreapp3.1/PactMultitargetTest.dll (.NETCoreApp,Version=v3.1)
Microsoft (R) Test Execution Command Line Tool Version 17.5.0 (x64)
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.

Passed!  - Failed:     0, Passed:     1, Skipped:     0, Total:     1, Duration: 428 ms - PactMultitargetTest.dll (netcoreapp3.1)
drew-cooper commented 1 year ago

Looking at the build artifacts on Ubuntu I see this:

$ ls bin/Debug/net472
Microsoft.VisualStudio.CodeCoverage.Shim.dll  Newtonsoft.Json.dll      PactNet.Abstractions.dll  nunit.engine.core.dll  pact_ffi.dll
NUnit3.TestAdapter.dll                        PactMultitargetTest.dll  PactNet.dll               nunit.engine.dll
NUnit3.TestAdapter.pdb                        PactMultitargetTest.pdb  nunit.engine.api.dll      nunit.framework.dll

Note the pact_ffi.dll which is the Windows native component.

The culprit appears to be this build/net461/PactNet.targets file in the Nuget package:

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <!-- This file supports .Net Framework, which doesn't automatically unpack the runtimes folder -->
  <ItemGroup>
    <Content Include="$(MSBuildThisFileDirectory)..\..\runtimes\win-x64\native\pact_ffi.dll">
      <Link>pact_ffi.dll</Link>
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
      <Visible>false</Visible>
    </Content>
  </ItemGroup>
</Project>

So, when building net461 or higher (in our case net472) we unconditionally copy in the Windows native component.

Manually copying in the Linux component fixes the test issue:

$ cp ~/.nuget/packages/pactnet/4.5.0/runtimes/linux-x64/native/libpact_ff
i.so bin/Debug/net472
$ dotnet test --no-build
Test run for /mnt/c/my-repo-dir/PactMultitargetTest/bin/Debug/net472/PactMultitargetTest.dll (.NETFramework,Version=v4.7.2)
Microsoft (R) Test Execution Command Line Tool Version 17.5.0 (x64)
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.

Passed!  - Failed:     0, Passed:     1, Skipped:     0, Total:     1, Duration: 595 ms - PactMultitargetTest.dll (net472)
Test run for /mnt/c/my-repo-dir/PactMultitargetTest/bin/Debug/netcoreapp3.1/PactMultitargetTest.dll (.NETCoreApp,Version=v3.1)
Microsoft (R) Test Execution Command Line Tool Version 17.5.0 (x64)
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.

Passed!  - Failed:     0, Passed:     1, Skipped:     0, Total:     1, Duration: 507 ms - PactMultitargetTest.dll (netcoreapp3.1)
drew-cooper commented 1 year ago

I would be happy to provide a PR to add similar platform detection as in PactNet.csproj to the PactNet.targets file to copy the appropriate binary for the system on which the tests are being built.

adamrodger commented 1 year ago

Currently .Net Framework is only supported on Windows, hence the unconditional copy in the targets file that you've found.

I've marked the issue as a feature request if Mono support is needed, but unsure on the relative priority of that given .Net Core has full Linux support and is the recommended framework going forwards.

drew-cooper commented 1 year ago

Thanks for the response. I'll have a workaround available to me soon so probably not that critical.