Response null instead of mocked resource

[tomsiwik]

Similar to pact-js I expected the response to be mocked so I could test an API client against pact's mock server. I modified example code to match my implementation as well. My codebase uses OpenAPI to generate the api client.

However I could not assert any of my tests due to the response being always null.

Reproduction steps with an OpenAPI generated petstore: https://github.com/tomsiwik/oas-pactnet-error

Now I'm not sure if this is csharp-netcore's client generator or that I'm using PactNet version 4.0.0-beta.3. Is this a bug or am I doing something wrong here? Thanks in advance.

[mefellows]

What do the logs say? If you could set them to DEBUG and share that would be very helpful.

[tomsiwik]

Added: LogLevel = PactLogLevel.Debug to PactConfig. I can't specify a log-file though.

# Just in case...
DEBUG=true dotnet test petstore

  Determining projects to restore...
  All projects are up-to-date for restore.
  Petstore -> /Users/tom/Code/problem/petstore/src/Petstore/bin/Debug/net6.0/Petstore.dll
  Petstore.Test -> /Users/tom/Code/problem/petstore/src/Petstore.Test/bin/Debug/net6.0/Petstore.Test.dll
Test run for /Users/tom/Code/problem/petstore/src/Petstore.Test/bin/Debug/net6.0/Petstore.Test.dll (.NETCoreApp,Version=v6.0)
Microsoft (R) Test Execution Command Line Tool Version 17.0.0+68bd10d3aee862a9fbb0bac8b3d474bc323024f3
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
[xUnit.net 00:00:01.07]     Petstore.Test.Api.PetApiTests.FindPetByStatus_WhenCalled_ReturnsPet [FAIL]
  Failed Petstore.Test.Api.PetApiTests.FindPetByStatus_WhenCalled_ReturnsPet [370 ms]
  Error Message:
   Expected pets not to be <null>.

With configuration:
- Use declared types and members
- Compare enums by value
- Include all non-private properties
- Include all non-private fields
- Match member by name (or throw)
- Without automatic conversion.
- Without automatic conversion.
- Be strict about the order of items in byte arrays

  Stack Trace:
     at FluentAssertions.Execution.XUnit2TestFramework.Throw(String message)
   at FluentAssertions.Execution.TestFrameworkProvider.Throw(String message)
   at FluentAssertions.Execution.CollectingAssertionStrategy.ThrowIfAny(IDictionary`2 context)
   at FluentAssertions.Equivalency.EquivalencyValidator.AssertEquality(EquivalencyValidationContext context)
   at FluentAssertions.Collections.CollectionAssertions`2.BeEquivalentTo[TExpectation](IEnumerable`1 expectation, Func`2 config, String because, Object[] becauseArgs)
   at FluentAssertions.Collections.CollectionAssertions`2.BeEquivalentTo(Object[] expectations)
   at Petstore.Test.Api.PetApiTests.<>c__DisplayClass2_0.<<FindPetByStatus_WhenCalled_ReturnsPet>b__0>d.MoveNext() in /Users/tom/Code/problem/petstore/src/Petstore.Test/Api/PetApiTests.cs:line 93
--- End of stack trace from previous location ---
   at PactNet.PactBuilder.VerifyAsync(Func`2 interact)
   at Petstore.Test.Api.PetApiTests.FindPetByStatus_WhenCalled_ReturnsPet() in /Users/tom/Code/problem/petstore/src/Petstore.Test/Api/PetApiTests.cs:line 87
--- End of stack trace from previous location ---
  Standard Output Messages:
 Mock server logs:

 [DEBUG][pact_mock_server::hyper_server] Creating pact request from hyper request
 [DEBUG][pact_mock_server::hyper_server] Extracting query from uri /pet/findByStatus?status=available
 [INFO][pact_mock_server::hyper_server] Received request HTTP Request ( method: GET, path: /pet/findByStatus, query: Some({"status": ["available"]}), headers: Some({"connection": ["Keep-Alive"], "user-agent": ["OpenAPI-Generator/1.0.0/csharp"], "accept-encoding": ["gzip", "deflate"], "host": ["127.0.0.1:49562"], "accept": ["application/json"]}), body: Empty )
 [INFO][pact_matching] comparing to expected HTTP Request ( method: GET, path: /pet/findByStatus, query: Some({"status": ["available"]}), headers: Some({"Accept": ["application/json"]}), body: Missing )
 [DEBUG][pact_matching]      body: ''
 [DEBUG][pact_matching]      matching_rules: MatchingRules { rules: {QUERY: MatchingRuleCategory { name: QUERY, rules: {} }, HEADER: MatchingRuleCategory { name: HEADER, rules: {} }, PATH: MatchingRuleCategory { name: PATH, rules: {} }} }
 [DEBUG][pact_matching]      generators: Generators { categories: {} }
 [DEBUG][pact_matching::matchers] String -> String: comparing '/pet/findByStatus' to '/pet/findByStatus' using Equality (false)
 [DEBUG][pact_matching] expected content type = '*/*', actual content type = '*/*'
 [DEBUG][pact_matching] content type header matcher = 'RuleList { rules: [], rule_logic: And, cascaded: false }'
 [DEBUG][pact_matching::matchers] String -> String: comparing 'available' to 'available' using Equality (false)
 [DEBUG][pact_matching] --> Mismatches: []
 [DEBUG][pact_mock_server::hyper_server] Test context = {"mockServer": Object({"href": String("http://127.0.0.1:49562"), "port": Number(49562)})}
 [INFO][pact_mock_server::hyper_server] Request matched, sending response HTTP Response ( status: 200, headers: Some({"Content-Type": ["application/json; charset=utf-8"]}), body: Present(16 bytes, application/json) )
 [DEBUG][pact_mock_server::hyper_server]      body: '[{"name":"Max"}]'

Failed!  - Failed:     1, Passed:    55, Skipped:     0, Total:    56, Duration: 398 ms - /Users/tom/Code/problem/petstore/src/Petstore.Test/bin/Debug/net6.0/Petstore.Test.dll (net6.0)

The assertion (inside async Task):

var example = new Pet(1, "Max", null, new List<string>());

await this.pact.VerifyAsync(async ctx =>
{
  var client = new PetApi(ctx.MockServerUri.ToString());

  List<Pet> pets = await client.FindPetsByStatusAsync("available"); // body: '[{"name":"Max"}]'

  pets.Should().BeEquivalentTo(new[] { example }); // Expected pets not to be <null>.
});

Hope this helps.

[mefellows]

OK thanks for sharing.

In the logs I can see:

 [INFO][pact_mock_server::hyper_server] Request matched, sending response HTTP Response ( status: 200, headers: Some({"Content-Type": ["application/json; charset=utf-8"]}), body: Present(16 bytes, application/json) )
 [DEBUG][pact_mock_server::hyper_server]      body: '[{"name":"Max"}]'

You can see it is returning a JSON body with the shape you defined in the test, so this tells me:

1. Your test setup is incorrect (not setting up the correct shaped mock response)
2. The API client doesn't know how to parse the response (probably because of 1)

[mefellows]

Given the Pet schema in the OAS document:

    Pet:
      x-swagger-router-model: io.swagger.petstore.model.Pet
      required:
        - name
        - photoUrls
      properties:
        id:
          type: integer
          format: int64
          example: 10
        name:
          type: string
          example: doggie
        category:
          $ref: '#/components/schemas/Category'
        photoUrls:
          type: array
          xml:
            wrapped: true
          items:
            type: string
            xml:
              name: photoUrl

Perhaps it's failing because the mandatory field photoUrls is absent?

[tomsiwik]

The test is a copy of and setup based on the tutorials from: https://github.com/pact-foundation/pact-net/blob/master/samples/EventApi/Consumer.Tests/EventsApiConsumerTests.cs#L63 (you can take this test as a reference)

photoUrls is an empty list: new List<string>() at the instantiation of new Pet(name, category, photoUrls).

I have received this error already with a different contract in my company. It's even simpler and requires only the name parameter. Due to NDA reasons I've recreated this error with a random OpenAPI spec I could find (petstore) so we have a reproducible repo.

Even a simple: https://github.com/pact-foundation/pact-net/blob/master/samples/ReadMe/Consumer.Tests/SomethingApiConsumerTests.cs#L56 returns null.

The same contracts work in Javascript and Java though and the test setup looks suspiciously similar. Yet it fails in .Net.

[mefellows]

Are you saying the example above doesn't work or your version that is adapted from it doesn't? I believe the examples are also run as part of CI, so they should be working. I've just tested them locally as well.

I'd suggest removing the generated OAS client from the equation, so that you can isolate the problem.

From the logs above, it is clear the mock server is returning [{"name":"Max"}] to your API client, so null is coming from your OAS client.

For clarity - I don't currently see any issues with the mock service which appears to be returning the correct response, and I think it's a bug in your code / test setup.

[mefellows]

Quick thought - you keep mentioning "at your company". Can I ask you test this on a non-company computer? Just to rule out any shenanigans related to corporate networks/laptops?

[tomsiwik]

Are you saying the example above doesn't work or your version that is adapted from it doesn't? I believe the examples are also run as part of CI, so they should be working. I've just tested them locally as well.

Perfect I haven't thought about building and running the examples. My bad.

I'd suggest removing the generated OAS client from the equation, so that you can isolate the problem.

Good Idea, I was suspecting the client to be the culprit as mentioned in my initial thoughts.

For clarity - I don't currently see any issues with the mock service which appears to be returning the correct response, and I think it's a bug in your code / test setup.

Okay I'll assume I did something wrong and retry a setup on a personal computer to make sure this is not FW related. Then prepare a simple client that is not using OAS generated code. And retry the test as is. If the tests work, I'll assume the generation of the OAS client has a bug somewhere. I'll retry to run the OAS client-generated code from the examples of OpenApi/generator and report the bug there. Hope this will not end up ping-ponging through libraries.

Thanks for the help, I'll try to investigate further.

[mefellows]

Hi @tomsiwik, I noticed you closed the issue. Did you get to the bottom of it?