laravel / framework

The Laravel Framework.
https://laravel.com
MIT License
32.45k stars 11k forks source link

AssertJsonFragment doesn't check fragments correctly #31212

Closed eithed closed 4 years ago

eithed commented 4 years ago

Description:

AssertJsonFragment seems to not care where the given same-node values are in the response tree. The assumption from the developer, when writing:

$response->assertJsonFragment([
    'A' => 'B',
    'E' => 'F',
]);

is that somewhere within the response there should be a fragment {'A': 'B', 'E': 'F'} (sure, the response can be {'H': {'I': 'J', 'E':'F', 'A':'B'}, 'G': 'C'}, but tested subset is within 'H'), however assertJsonFragment checks for existence of {'A': 'B'} and {'E': 'F'} anywhere in the response altogether.

Steps To Reproduce:

Create route as:

Route::get('/test', function(){
    return response()->json(['data' => [
    [
        'A' => 'B',
        'C' => 'D'
    ], [
        'E' => 'F',
        'G' => 'H'
    ]]]);
});

and unit test as:

<?php

namespace Tests\Feature;

use Illuminate\Validation\ValidationException;
use Tests\TestCase;

class TestTest extends TestCase
{
    public function test_it_shows_the_issue()
    {
        $response = $this->json('get', '/test');

        $response->assertJsonFragment([
            'A' => 'B',
            'E' => 'F',
        ]);
    }
}

Expected result: test should fail Current result: test passes

If we update the route to be:

Route::get('/test', function(){
    return response()->json(['data' => [
    [
        'A' => 'B',
        'C' => 'D'
    ], [[[[[[
        'E' => 'F',
        'G' => 'H'
    ]]]]]]]]);
});

the test still passes

driesvints commented 4 years ago

This is the correct behavior. The testing method just asserts that a given fraction is present within the response, regardless of its nesting.

eithed commented 4 years ago

@driesvints - I'd disagree that my response contains fragment 'A': 'B', 'E': 'F'. It contains fragments 'A': 'B' and 'E': 'F'. I was caught completely off-guard by this behaviour and I've not seen this documented anywhere.

driesvints commented 4 years ago

@eithed if you look into the method itself you notice that it recursively searches over each entry you provide:

https://github.com/laravel/framework/blob/6.x/src/Illuminate/Foundation/Testing/TestResponse.php#L549

If I update your test method to below it fails btw:

        $response = $this->json('get', '/test');

        $response->assertJsonFragment([
            [
                'A' => 'B',
                'E' => 'F',
            ],
        ]);
driesvints commented 4 years ago

Maybe this can be documented better though. Feel free to send in a PR.

eithed commented 4 years ago

@driesvints I've tried performing the assertion you've provided before submitting the ticket. It unfortunately doesn't do the job, as it does a strict comparison, so, for example response {'A': 'B', 'C': 'D', 'E': 'F'} fails that assertion with message:

Unable to find JSON fragment: 

[[{"A":"B","E":"F"}]]

within

[{"data":[{"A":"B","C":"D","E":"F"}]}].

even though response does contain given fragment.

I think this conversation is good enough for documenting use cases, and given that I'm first person to find problem with this behaviour, it might as well be my understanding of this method.

dani0332 commented 1 month ago

@eithed No, you're not alone this. I spent hours grappling with this as well. Trying to make sense of how it works due to lack of documentation on use cases. But this really thread really clear lot of things up.

For anyone searching like me

It basically searches each element (root level) of your assertion regardless of its level in the response but the catch is (and the part that it confuses IMO) that it tries to match it exactly, for it to be considered as true. So with below response

[
  {
    "data": {
      "id": 1,
      "invoice_payor": null,
      "license_id": 1,
      "manager_address_book_no": "8208266",
      "manager_first_name": "Ciaran",
      "manager_last_name": "Byers",
      "manager_phone_number": "+1-115-620-5198",
      "payment_methods": [
        {
          "account_id": 1,
          "address": {
            "account_id": 1,
            "address_book_no": "",
            "address_line_1": "",
            "address_line_2": "SMCHS",
            "city": "",
            "created_at": "2024-09-20T11:35:00.000000Z",
            "duns_number": "",
            "ext": "",
            "federal_id_number": "",
            "id": 7,
            "manager_address_book_no": "",
            "organization_name": "Edited BA CC",
            "parent_ab_number": "",
            "phone": null,
            "stab": null,
            "state": 34,
            "type": "billing",
            "updated_at": "2024-09-20T11:35:04.000000Z",
            "zip_code": "00501"
          },
          "address_id": 7,
          "created_at": "09/20/2024",
          "credit_card": {
            "exp_month": "10",
            "exp_year": "2026",
            "id": 1,
            "last_four": "1111",
            "payment_method_id": 3,
            "platform": "CCM",
            "status": 1
          },
          "id": 3,
          "is_default": 0,
          "net_term_status": 0,
          "nick_name": "Edited Credit Card l8",
          "tenants": [],
          "type": "Credit Card",
          "updated_at": "09/20/2024",
          "user_id": 3
        },
        {
          "account_id": 1,
          "address": {
            "account_id": 1,
            "address_book_no": "",
            "address_line_1": "805 15TH ST, NW",
            "address_line_2": "SMCHS",
            "city": "Holtsville",
            "created_at": "2024-09-20T11:36:10.000000Z",
            "duns_number": "34-343-4543",
            "ext": "",
            "federal_id_number": "",
            "id": 10,
            "manager_address_book_no": "",
            "organization_name": "NT Arpatech312",
            "parent_ab_number": "",
            "phone": null,
            "stab": null,
            "state": 34,
            "type": "billing",
            "updated_at": "2024-09-20T11:36:10.000000Z",
            "zip_code": "77084"
          },
          "address_id": 10,
          "created_at": "09/20/2024",
          "credit_card": null,
          "id": 5,
          "is_default": 0,
          "net_term_status": 0,
          "nick_name": "Net Terms r5",
          "tenants": [],
          "type": "Net Terms",
          "updated_at": "09/20/2024",
          "user_id": 3
        }
      ],
      "restricted_service_providers": [],
      "sales_account_id": null,
      "sales_division": "",
      "status": "Active",
      "subscription_plan_id": 1,
      "updated_at": "09/20/2024",
      "web_group": null
    },
    "message": "",
    "status_code": 200,
    "success": true
  }
]

and with below assertion, it gets passed (notice the manager_first_name is on another level relative to credit_card but it still gets through)

$this->response->assertJsonFragment([
      "credit_card" => [
          "exp_month" => "10",
          "exp_year" => "2026",
          "id" => 1,
          "last_four" => "1111",
          "payment_method_id" => 3,
          "platform" => "CCM",
          "status" => 1,
      ],
      "manager_first_name" => "Ciaran"
]);

Also all of the (root level) elements needs to be found so below fails

$this->response->assertJsonFragment([
      "credit_card" => [
          "exp_month" => "10",
          "exp_year" => "2026",
          "id" => 1,
          "last_four" => "1111",
          "payment_method_id" => 3,
          "platform" => "CCM",
          "status" => 1,
      ],
      "manager_first_name__xx" => "Ciaran" // added __xx 
/
]);

Below also gets failed since, now manager_first_name is not root level and you need to mention entire object (that should have been present in response) inside data, to be able to get a match

$this->response->assertJsonFragment([
    'data' => [
        // complete the object exactly
        "manager_first_name" => "Ciaran"
    ]
]);

Similar case of as above, but with array. This also fails because of root element doesn't contain the entire object (since It matches exactly every element)

  $this->response->assertJsonFragment([
      'payment_methods' => [
         // you'll need to complete the object entirely since its a level down the root
          "credit_card" => [
              "exp_month" => "10",
              "exp_year" => "2026",
              "id" => 1,
              "last_four" => "1111",
              "payment_method_id" => 3,
              "platform" => "CCM",
              "status" => 1,
          ]
      ]
  ]);