NUnit Framework Exception

volkovivn commented 10 months ago

Hi team,

When try to use PactNet with NUnit, an exception is thrown. I'm not quite sure if it is a problem with NUnit or PactNet. I wasn't able to find the solution by myself. Could anyone help me? It is pretty easy to reproduce:

Replace the framework from XUnit to NUnit. (for reference you test it on Consumer.Tests.csproj)
Run all tests
Only first test in class will pass, exception is thrown for rest 😞

Exception:

PactNet.Exceptions.PactFailureException : Unable to perform the given action. The interop call indicated failure
   at PactNet.Drivers.InteropActionExtensions.CheckInteropSuccess(Boolean success) in D:\Repos\pact-net\src\PactNet\Drivers\InteropActionExtensions.cs:line 19
   at PactNet.Drivers.HttpInteractionDriver.GivenWithParam(String description, String name, String value) in D:\Repos\pact-net\src\PactNet\Drivers\HttpInteractionDriver.cs:line 41
   at PactNet.RequestBuilder.Given(String providerState, IDictionary`2 parameters) in D:\Repos\pact-net\src\PactNet\RequestBuilder.cs:line 424
   at PactNet.RequestBuilder.PactNet.IRequestBuilderV4.Given(String providerState, IDictionary`2 parameters) in D:\Repos\pact-net\src\PactNet\RequestBuilder.cs:line 291
   at Consumer.Tests.OrdersClientTests.GetOrderAsync_WhenCalled_ReturnsOrder() in D:\Repos\pact-net\samples\OrdersApi\Consumer.Tests\OrdersClientTests.cs:line 46
   at NUnit.Framework.Internal.TaskAwaitAdapter.GenericAdapter`1.GetResult()
   at NUnit.Framework.Internal.AsyncToSyncAdapter.Await(Func`1 invoke)
   at NUnit.Framework.Internal.Commands.TestMethodCommand.RunTestMethod(TestExecutionContext context)
   at NUnit.Framework.Internal.Commands.TestMethodCommand.Execute(TestExecutionContext context)
   at NUnit.Framework.Internal.Execution.SimpleWorkItem.<>c__DisplayClass4_0.<PerformWork>b__0()
   at NUnit.Framework.Internal.ContextUtils.<>c__DisplayClass1_0`1.<DoIsolated>b__0(Object _)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at NUnit.Framework.Internal.ContextUtils.DoIsolated(ContextCallback callback, Object state)
   at NUnit.Framework.Internal.ContextUtils.DoIsolated[T](Func`1 func)
   at NUnit.Framework.Internal.Execution.SimpleWorkItem.PerformWork()

-----

One or more child tests had errors
  Exception doesn't have a stacktrace

YOU54F commented 10 months ago

Hey Hey,

Would you care to provide a repro? You can just fork the repo, make the updates and show the failure in a PR, it would help others to resolve 👍🏾

phil-rice commented 9 months ago

Hi.

I am getting the same issue. I raised a stack over flow question https://stackoverflow.com/questions/77196218/i-have-a-interop-exception-with-a-consumer-contract-test-under-dotnet

In it I reference a sample repo that demonstrates the problem.

Git: https://github.com/phil-rice/CandidateSourcing
Tag: PactNotWorking
Test: project casClient, file: cas.pact.consumer.test.cs

Would changing to XUnit get rid of this defect for me as a workaround?

adamrodger commented 9 months ago

You're using a constructor to initialise the pact, which is the correct approach when using xUnit because each test gets its own instance of the test class.

If I remember correctly, and it's been a long time since I've used NUnit, you need to provide a special one time setup method instead of a constructor to ensure setup code only runs once.

The error will be because it's tried to initialise pact again whilst in the middle of running a pact with the same provider and consumer names, which would attempt to create a duplicate.

Edit: see the NUnit docs here:

https://docs.nunit.org/articles/nunit/writing-tests/attributes/onetimesetup.html

phil-rice commented 9 months ago

Thank you for that. Sorry it took a while responding I was on leave.

Unfortunately this doesn't seem to fix it. It is entirely possible I have messed up of course.

Here is the git repo (I simplified it a lot so you need to clone it again if you have the old one around) git@github.com:phil-rice/CandidateSourcing.git And there is a tag PactIssue that demonstrates the issue

The pact consumer test is in the tests project with the name jobclinet.consumer.pact.test.cs

The error I get is

 EnsureCanReadOneJob
   Source: jobclient.consumer.pact.test.cs line 61
   Duration: 18 ms

  Message: 
PactNet.Exceptions.PactFailureException : Unable to perform the given action. The interop call indicated failure`

The relevant code is below. As you can see I do the initialisation in the init method which is marked with [OneTimeSetup]. The first pact test runs fine. If I comment out the first test, the second one fails but it fails because I haven't finished it yet: i.e. it is an example of a failing pact test rather than an interop exception.

 public class JobClientConsumerPactTest
 {

     private IPactBuilderV2 pact;
     private readonly int port = 9222;
     private JobClient client;

     [OneTimeSetUp]
     public void Init()
     {
         var httpClient = new System.Net.Http.HttpClient();
         var jobSettings = new JobSettings { BaseUrl = $"http://localhost:{port}/api/" };
         this.client = new JobClient(httpClient, new OptionsWrapper<JobSettings>(jobSettings));

         var config = new PactConfig
         {
             PactDir = @"..\..\..\..\artifacts\pacts"
         };

         IPactV2 pact = Pact.V2("JobClient", "JobApi", config);

         this.pact = pact.WithHttpInteractions(port);
     }

     [Test]
     async public Task EnsureCanReadEmptyJobs()
     {
         pact
           .UponReceiving("A request for jobs")
           .Given("No items")
           .WithRequest(HttpMethod.Get, "/api/Job")
           .WithQuery("eagerLoad", "False")
           .WillRespond()
           .WithStatus(200)
           .WithBody("[]", "application/json");

         await pact.VerifyAsync(async ctx =>
         {
             var result = await client.GetAllAsync();
             Assert.AreEqual(0, result.Count());

         });

     }
     [Test]
     async public Task EnsureCanReadOneJob()
     {
         var guid = Guids.from("1");
         pact
           .UponReceiving("A request for a single job")
           .Given("Items exist")
           .WithRequest(HttpMethod.Get, $"/api/Job/{guid}")
           .WillRespond()
           .WithStatus(200)
           .WithBody("[]", "application/json");

         await pact.VerifyAsync(async ctx =>
         {
             var result = await client.GetByIdAsync(guid);
             Assert.AreEqual(guid, result.Id);

         });
     }
 }

adamrodger commented 9 months ago

Can you attach the logs and the full stack trace?

Also, please could you try a basic example as shown in the ReadMe as a starter? You're doing a few non-standard things all at the same time there, such as manually overriding the port of the mock server, and obviously using a different test framework.

It's almost certainly the mock server port override because you're not shutting down the mock server between tests, and thus the port will be occupied. It's probably worth following the examples in the ReadMe and the linked workshop to get that working first before you try a non-standard setup.

phil-rice commented 9 months ago

OK I've tried quite hard to make it this library work with NUnit and I am failing.

I have of course stopped changing the port. What else am I doing wrong in my 'non standard setup'. Would it be possible for you to make one of those tests work? Or tell me how to do it, Or give a sample NUnit test with more more than one pact in it?

DavidJFowler commented 8 months ago

@phil-rice Try changing [OneTimeSetUp] to [SetUp] and also ensure that your client is disposed after running each test:

public class JobClientConsumerPactTest
{
      /* ...  */
     [Test]
     async public Task EnsureCanReadEmptyJobs()
     {
         pact
           .UponReceiving("A request for jobs")
           .Given("No items")
           .WithRequest(HttpMethod.Get, "/api/Job")
           .WithQuery("eagerLoad", "False")
           .WillRespond()
           .WithStatus(200)
           .WithBody("[]", "application/json");

         await pact.VerifyAsync(async ctx =>
         {
             // create the client here
             using var httpClient = new System.Net.Http.HttpClient();
             var jobSettings = new JobSettings { BaseUrl = $"http://localhost:{port}/api/" };
             var client = new JobClient(httpClient, new OptionsWrapper<JobSettings>(jobSettings));
             var result = await client.GetAllAsync();
             Assert.AreEqual(0, result.Count());
         });

        /* ...  */

     }

This works for me, with or without a fixed port number.

adamrodger commented 8 months ago

I've just tried this out with the sample project and it works fine. All I did was:

Uninstall Xunit and install NUnit
Add the [TestFixture] attribute to the class
Change the constructor to a [SetUp] method
Remove the XUnit logger
Rename Fact to Test on each test

The full example is here:

[TestFixture]
public class OrdersClientTests
{
    private IPactBuilderV4 pact;
    private Mock<IHttpClientFactory> mockFactory;

    [SetUp]
    public void Setup()
    {
        this.mockFactory = new Mock<IHttpClientFactory>();

        var config = new PactConfig
        {
            PactDir = "../../../pacts/",
            DefaultJsonSettings = new JsonSerializerSettings
            {
                ContractResolver = new CamelCasePropertyNamesContractResolver(),
                Converters = new JsonConverter[] { new StringEnumConverter() }
            },
            LogLevel = PactLogLevel.Debug
        };

        this.pact = Pact.V4("Fulfilment API", "Orders API", config).WithHttpInteractions();
    }

    [Test]
    public async Task GetOrderAsync_WhenCalled_ReturnsOrder()
    {
        var expected = new OrderDto(1, OrderStatus.Pending, new DateTimeOffset(2023, 6, 28, 12, 13, 14, TimeSpan.FromHours(1)));

        this.pact
            .UponReceiving("a request for an order by ID")
                .Given("an order with ID {id} exists", new Dictionary<string, string> { ["id"] = "1" })
                .WithRequest(HttpMethod.Get, "/api/orders/1")
                .WithHeader("Accept", "application/json")
            .WillRespond()
                .WithStatus(HttpStatusCode.OK)
                .WithJsonBody(new
                {
                    Id = Match.Integer(expected.Id),
                    Status = Match.Regex(expected.Status.ToString(), string.Join("|", Enum.GetNames<OrderStatus>())),
                    Date = Match.Type(expected.Date.ToString("O"))
                });

        await this.pact.VerifyAsync(async ctx =>
        {
            this.mockFactory
                .Setup(f => f.CreateClient("Orders"))
                .Returns(() => new HttpClient
                {
                    BaseAddress = ctx.MockServerUri,
                    DefaultRequestHeaders =
                    {
                        Accept = { MediaTypeWithQualityHeaderValue.Parse("application/json") }
                    }
                });

            var client = new OrdersClient(this.mockFactory.Object);

            OrderDto order = await client.GetOrderAsync(1);

            order.Should().Be(expected);
        });
    }

    [Test]
    public async Task GetOrderAsync_UnknownOrder_ReturnsNotFound()
    {
        this.pact
            .UponReceiving("a request for an order with an unknown ID")
                .WithRequest(HttpMethod.Get, "/api/orders/404")
                .WithHeader("Accept", "application/json")
            .WillRespond()
                .WithStatus(HttpStatusCode.NotFound);

        await this.pact.VerifyAsync(async ctx =>
        {
            this.mockFactory
                .Setup(f => f.CreateClient("Orders"))
                .Returns(() => new HttpClient
                {
                    BaseAddress = ctx.MockServerUri,
                    DefaultRequestHeaders =
                    {
                        Accept = { MediaTypeWithQualityHeaderValue.Parse("application/json") }
                    }
                });

            var client = new OrdersClient(this.mockFactory.Object);

            Func<Task> action = () => client.GetOrderAsync(404);

            var response = await action.Should().ThrowAsync<HttpRequestException>();
            response.And.StatusCode.Should().Be(HttpStatusCode.NotFound);
        });
    }

    [Test]
    public async Task UpdateOrderAsync_WhenCalled_UpdatesOrder()
    {
        this.pact
            .UponReceiving("a request to update the status of an order")
                .Given("an order with ID {id} exists", new Dictionary<string, string> { ["id"] = "1" })
                .WithRequest(HttpMethod.Put, "/api/orders/1/status")
                .WithJsonBody(Match.Regex(OrderStatus.Fulfilling.ToString(), string.Join("|", Enum.GetNames<OrderStatus>())))
            .WillRespond()
                .WithStatus(HttpStatusCode.NoContent);

        await this.pact.VerifyAsync(async ctx =>
        {
            this.mockFactory
                .Setup(f => f.CreateClient("Orders"))
                .Returns(() => new HttpClient
                {
                    BaseAddress = ctx.MockServerUri
                });

            var client = new OrdersClient(this.mockFactory.Object);

            await client.UpdateOrderAsync(1, OrderStatus.Fulfilling);
        });
    }
}

All of these tests pass and the pact file is produced as expected:

pact-foundation / pact-net

NUnit Framework Exception #471