Cats endlessly looping on cyclic references

[molmar-form3]

Cats keeps looping endlessly through creating payloads when there are cyclical references in the swagger.

An example that stalls fuzzing:

basePath: /v1
consumes:
- application/json
- application/vnd.api+json
definitions:
  CatFoodReturn:
    properties:
      attributes:
        properties:
          charges_amount:
            $ref: '#/definitions/CatFoodReturn'
            x-omitempty: true
            x-parent: data.attributes
          clearing_id:
            description: Unique identifier for organisations collecting payments
            example: '123456'
            type: string
            x-parent: data.attributes
          compensation_amount:
            $ref: '#/definitions/CatFoodReturn'
            x-omitempty: true
            x-parent: data.attributes
          processing_date:
            description: 'Date on which the operation is to be debited from the debtor
              account. Formatted according to ISO 8601 format: YYYY-MM-DD.'
            example: '2015-02-12'
            format: date
            type: string
            x-nullable: true
            x-parent: data.attributes
          return_amount:
            $ref: '#/definitions/CatFoodReturn'
            x-omitempty: true
            x-parent: data.attributes
          return_code:
            type: string
            x-parent: data.attributes
          return_initiator:
            enum:
            - FOODBANK
            - CUSTOMER
            type: string
            x-parent: data.attributes
          scheme_processing_date:
            description: 'Date on which the operation is processed by the scheme.
              Formatted according to ISO 8601 format: YYYY-MM-DD. Only used if different
              from `processing_date`.'
            example: '2015-02-12'
            format: date
            type: string
            x-nullable: true
            x-parent: data.attributes
          scheme_transaction_id:
            type: string
            x-parent: data.attributes
        type: object
        x-parent: data
      created_on:
        format: date-time
        type: string
        x-nullable: true
      id:
        format: uuid
        type: string
      modified_on:
        format: date-time
        type: string
        x-nullable: true
      organisation_id:
        format: uuid
        type: string
      relationships:
        properties:
          direct_debit:
            properties:
              data:
                type: array
            type: object
          direct_debit_return_admission:
            properties:
              data:
                type: array
            type: object
          direct_debit_return_reversal:
            properties:
              data:
                type: array
            type: object
          direct_debit_return_submission:
            properties:
              data:
                items:
                  $ref: '#/definitions/CatFoodReturnSubmission'
                type: array
            type: object
        type: object
      type:
        pattern: ^[A-Za-z_]*$
        type: string
      version:
        minimum: 0
        type: integer
    required:
    - id
    - organisation_id
    - attributes
    type: object
    x-access:
    - Public
  CatFoodReturnSubmission:
    properties:
      attributes:
        properties:
          scheme_status_code:
            type: string
            x-parent: data.attributes
          scheme_status_code_description:
            type: string
            x-parent: data.attributes
          status_reason:
            type: string
            x-parent: data.attributes
          submission_datetime:
            format: date-time
            readOnly: true
            type: string
            x-parent: data.attributes
          transaction_start_datetime:
            format: date-time
            readOnly: true
            type: string
            x-parent: data.attributes
        type: object
        x-parent: data
      created_on:
        format: date-time
        type: string
        x-nullable: true
      id:
        format: uuid
        type: string
      modified_on:
        format: date-time
        type: string
        x-nullable: true
      organisation_id:
        format: uuid
        type: string
      relationships:
        properties:
          direct_debit:
            properties:
              data:
                type: array
            type: object
          direct_debit_return:
            properties:
              data:
                items:
                  $ref: '#/definitions/CatFoodReturn'
                type: array
            type: object
        type: object
      type:
        pattern: ^[A-Za-z_]*$
        type: string
      version:
        minimum: 0
        type: integer
    required:
    - id
    - organisation_id
    type: object
    x-access:
    - Public
host: api.foodCorp.tech
info:
  title: FoodCorp Public API
  version: '1'
parameters:
  admissionIdParam:
    description: Cat Food Admission Id
    format: uuid
    in: path
    name: admissionId
    required: true
    type: string
  decisionIdParam:
    description: Cat Food decision id
    format: uuid
    in: path
    name: decisionId
    required: true
    type: string
  catFoodIdParam:
    description: Cat Food Id
    format: uuid
    in: path
    name: id
    required: true
    type: string
  recallIdParam:
    description: Recall Id
    format: uuid
    in: path
    name: recallId
    required: true
    type: string
  returnIdParam:
    description: Return Id
    format: uuid
    in: path
    name: returnId
    required: true
    type: string
  reversalIdParam:
    description: Reversal Id
    format: uuid
    in: path
    name: reversalId
    required: true
    type: string
  submissionIdParam:
    description: Cat Food decision submission id
    format: uuid
    in: path
    name: submissionId
    required: true
    type: string
paths:
  /transaction/catfoods/{id}/returns:
    post:
      consumes:
      - application/vnd.api+json
      - application/json
      parameters:
      - $ref: '#/parameters/catFoodIdParam'
      - in: body
        name: Return creation request
        schema:
          $ref: '#/definitions/CatFoodReturn'
      responses:
        201:
          description: Return creation response
          schema:
            $ref: '#/definitions/CatFoodReturn'
        400:
          description: Return creation error
          schema:
            $ref: '#/definitions/CatFoodReturn'
      summary: Create direct debit return
      tags:
      - CatFoods
      x-access:
      - Public
  /transaction/catfoods/{id}/returns/{returnId}:
    get:
      parameters:
      - $ref: '#/parameters/catFoodIdParam'
      - $ref: '#/parameters/returnIdParam'
      responses:
        200:
          description: Return details
          schema:
            $ref: '#/definitions/CatFoodReturn'
      summary: Fetch direct debit return
      tags:
      - CatFoods
      x-access:
      - Public
  /transaction/catfoods/{id}/returns/{returnId}/submissions:
    post:
      consumes:
      - application/vnd.api+json
      - application/json
      parameters:
      - $ref: '#/parameters/catFoodIdParam'
      - $ref: '#/parameters/returnIdParam'
      - in: body
        name: Return submission creation request
        schema:
          $ref: '#/definitions/CatFoodReturn'
      responses:
        201:
          description: Return submission creation response
          schema:
            $ref: '#/definitions/CatFoodReturn'
        400:
          description: Return submission creation error
          schema:
            $ref: '#/definitions/CatFoodReturn'
      summary: create direct debit return submission
      tags:
      - CatFoods
      x-access:
      - Public
  /transaction/catfoods/{id}/returns/{returnId}/submissions/{submissionId}:
    get:
      parameters:
      - $ref: '#/parameters/catFoodIdParam'
      - $ref: '#/parameters/returnIdParam'
      - $ref: '#/parameters/submissionIdParam'
      responses:
        200:
          description: Return submission details
          schema:
            $ref: '#/definitions/CatFoodReturn'
      summary: Fetch return submission
      tags:
      - CatFoods
      x-access:
      - Public
produces:
- application/vnd.api+json
- application/json
responses:
  BadGateway:
    description: Bad Gateway
  BadRequest:
    description: Bad Request
  Conflict:
    description: Conflict
  Forbidden:
    description: Action Forbidden
  NotFound:
    description: Not Found
  UnexpectedError:
    description: Unexpected Error
schemes:
- https
security:
- OAuth2: []
securityDefinitions:
  Basic:
    type: basic
  OAuth2:
    description: OAuth 2.0 with Client Credentials Grant type
    flow: application
    tokenUrl: /oauth2/token
    type: oauth2
swagger: '2.0'

cats --contract=bad.yaml --server=https://api.foodcorp.co -D

Output:

[**********][*******] 👣 trace     Resolving model 'charges_amount' to example
[**********][*******] 👣 trace     Schema properties not null charges_amount: [attributes, created_on, id, modified_on, organisation_id, relationships, type, version]
[**********][*******] 👣 trace     Creating example from model values charges_amount
[**********][*******] 👣 trace     Resolving model 'attributes' to example
[**********][*******] 👣 trace     Schema properties not null attributes: [charges_amount, clearing_id, compensation_amount, processing_date, return_amount, return_code, return_initiator, scheme_processing_date, scheme_transaction_id]
[**********][*******] 👣 trace     Creating example from model values attributes
[**********][*******] 👣 trace     Resolving model 'charges_amount' to example
[**********][*******] 👣 trace     Schema properties not null charges_amount: [attributes, created_on, id, modified_on, organisation_id, relationships, type, version]
[**********][*******] 👣 trace     Creating example from model values charges_amount
[**********][*******] 👣 trace     Resolving model 'attributes' to example
[**********][*******] 👣 trace     Schema properties not null attributes: [charges_amount, clearing_id, compensation_amount, processing_date, return_amount, return_code, return_initiator, scheme_processing_date, scheme_transaction_id]
[**********][*******] 👣 trace     Creating example from model values attributes
[**********][*******] 👣 trace     Resolving model 'charges_amount' to example
[**********][*******] 👣 trace     Schema properties not null charges_amount: [attributes, created_on, id, modified_on, organisation_id, relationships, type, version]
[**********][*******] 👣 trace     Creating example from model values charges_amount
[**********][*******] 👣 trace     Resolving model 'attributes' to example

Hacky python script to detect cyclical references:

import networkx as nx
import sys

#python3 display.py swagger.yaml

gr = nx.DiGraph()

file = open(sys.argv[1],'r')

dot = ""
defs = False

for x in file.readlines():
    if "definitions:" in x:
        defs = True
        continue
    if defs and x.startswith("  ") and x[2] != ' ':
        dot = x.strip()[:-1]
    if defs and "$ref" in x:
        other = x.split('/')[2].strip()[:-1]
        gr.add_edges_from([(dot, other)])
    if defs and x[0] != ' ':
        defs = False
        break

print(nx.is_directed_acyclic_graph(gr))
for x in list(nx.simple_cycles(gr)):
    print(x)

% python3 display.py bad.yaml
False
['CatFoodReturn']
['CatFoodReturnSubmission', 'CatFoodReturn']

[en-milie]

Hi @molmar-form3. Thank you for raising this. I have a fix almost ready.

[en-milie]

This is now fixed in https://github.com/Endava/cats/commit/74acf24c9b2afac5913d0b9ccf9c65c51b08ee74 and will be available in 7.2.2 later this week.

[molmar-form3]

Looks good, thanks for patching this so quick, will give it a try later this week:)

[en-milie]

The fix is part of https://github.com/Endava/cats/releases/tag/cats-7.3.0

[molmar-form3]

@en-milie Works perfect, thanks for the quick fix.