Recursive components

[chriswhite199]

From 1.8+, there is an issue with openapi specs that contain recursive or cyclic references in components.

See attached sample spec and test file to reproduce. Works without issue for 1.7.0, but 1.8.0 yields a python error related to maximum recursion depth

test_recursive.py

from openapi3 import OpenAPI
import os
import yaml

test_dir = os.path.dirname(__file__)

def test_recursive_spec():
    with open(test_dir + '/recursive-spec.yaml') as f:
        yaml_spec = yaml.safe_load(f.read())

    spec = OpenAPI(yaml_spec)

recursive-spec.yaml

openapi: 3.0.0
info:
  version: 1.0.0
  title: Example API
  description: A sample API to illustrate Recursive OpenAPI components
paths:
  /list:
    get:
      description: Returns a list of objects              
      responses:
        '200':
          description: Success
          content:
            'application/json':
              schema:
                $ref: '#/components/schemas/MyObject'
components:
  schemas:
    MyObject:
      type: object
      properties:
        name:
          type: string
        child:
          $ref: '#/components/schemas/MyObject'

Output

Output from 1.7.0 and 1.8.0 parsing the same yaml spec

(.venv) cswhite@xps17:~/workspace/openapi3$ pip list
Package            Version
------------------ ---------
certifi            2023.7.22
charset-normalizer 3.3.0
exceptiongroup     1.1.3
idna               3.4
iniconfig          2.0.0
openapi3           1.7.0
packaging          23.2
pip                22.0.2
pluggy             1.3.0
pytest             7.4.2
PyYAML             6.0.1
requests           2.31.0
setuptools         59.6.0
tomli              2.0.1
urllib3            2.0.7

(.venv) cswhite@xps17:~/workspace/openapi3$ pytest --rootdir=tests
========================================================================================= test session starts ==========================================================================================
platform linux -- Python 3.10.12, pytest-7.4.2, pluggy-1.3.0
rootdir: /home/cswhite/workspace/openapi3/tests
collected 1 item                                                                                                                                                                                       

tests/test_recursive_spec.py .                                                                                                                                                                   [100%]

========================================================================================== 1 passed in 0.11s ===========================================================================================

(.venv) cswhite@xps17:~/workspace/openapi3$ pip install openapi3==1.8.2
Collecting openapi3==1.8.2
  Using cached openapi3-1.8.2-py2.py3-none-any.whl (27 kB)
Requirement already satisfied: requests in ./.venv/lib/python3.10/site-packages (from openapi3==1.8.2) (2.31.0)
Requirement already satisfied: PyYaml in ./.venv/lib/python3.10/site-packages (from openapi3==1.8.2) (6.0.1)
Requirement already satisfied: idna<4,>=2.5 in ./.venv/lib/python3.10/site-packages (from requests->openapi3==1.8.2) (3.4)
Requirement already satisfied: charset-normalizer<4,>=2 in ./.venv/lib/python3.10/site-packages (from requests->openapi3==1.8.2) (3.3.0)
Requirement already satisfied: certifi>=2017.4.17 in ./.venv/lib/python3.10/site-packages (from requests->openapi3==1.8.2) (2023.7.22)
Requirement already satisfied: urllib3<3,>=1.21.1 in ./.venv/lib/python3.10/site-packages (from requests->openapi3==1.8.2) (2.0.7)
Installing collected packages: openapi3
  Attempting uninstall: openapi3
    Found existing installation: openapi3 1.7.0
    Uninstalling openapi3-1.7.0:
      Successfully uninstalled openapi3-1.7.0
Successfully installed openapi3-1.8.2

(.venv) cswhite@xps17:~/workspace/openapi3$ pytest --rootdir=tests
========================================================================================= test session starts ==========================================================================================
platform linux -- Python 3.10.12, pytest-7.4.2, pluggy-1.3.0
rootdir: /home/cswhite/workspace/openapi3/tests
collected 1 item                                                                                                                                                                                       

tests/test_recursive_spec.py F                                                                                                                                                                   [100%]

=============================================================================================== FAILURES ===============================================================================================
_________________________________________________________________________________________ test_recursive_spec __________________________________________________________________________________________

    def test_recursive_spec():
        with open(test_dir + '/recursive-spec.yaml') as f:
            yaml_spec = yaml.safe_load(f.read())

>       spec = OpenAPI(yaml_spec)

tests/test_recursive_spec.py:11: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.venv/lib/python3.10/site-packages/openapi3/openapi.py:60: in __init__
    super(OpenAPI, self).__init__([], raw_document, self)
.venv/lib/python3.10/site-packages/openapi3/object_base.py:129: in __init__
    self._parse_data()
.venv/lib/python3.10/site-packages/openapi3/openapi.py:187: in _parse_data
    self._resolve_references()
.venv/lib/python3.10/site-packages/openapi3/object_base.py:524: in _resolve_references
    value._resolve_references()
.venv/lib/python3.10/site-packages/openapi3/object_base.py:524: in _resolve_references
    value._resolve_references()
.venv/lib/python3.10/site-packages/openapi3/object_base.py:662: in _resolve_references
    value._resolve_references()
.venv/lib/python3.10/site-packages/openapi3/object_base.py:524: in _resolve_references
    value._resolve_references()
.venv/lib/python3.10/site-packages/openapi3/object_base.py:662: in _resolve_references
    value._resolve_references()
.venv/lib/python3.10/site-packages/openapi3/object_base.py:524: in _resolve_references
    value._resolve_references()
E   RecursionError: maximum recursion depth exceeded while calling a Python object
!!! Recursion detected (same locals & position)
======================================================================================= short test summary info ========================================================================================
FAILED tests/test_recursive_spec.py::test_recursive_spec - RecursionError: maximum recursion depth exceeded while calling a Python object
========================================================================================== 1 failed in 0.20s ===========================================================================================

[commonism]

c.f. https://github.com/commonism/aiopenapi3/blob/master/tests/fixtures/schema-recursion.yaml

[caseyoneill]

I am seeing this exact problem while attempting to parse the Jira API: https://dac-static.atlassian.com/cloud/jira/platform/swagger-v3.v3.json

[commonism]

Using aiopenapi3 instead, there is still some issues with the jira api description document. It

• defines enums for an array type (NotImplementedError)
• members of the discriminated unions do not use const/enums to declare the discriminator key (DiscriminatorWarning),

but this is addressable.

import pytest

import aiopenapi3.errors
from aiopenapi3 import OpenAPI
from aiopenapi3.plugin import Init
from aiopenapi3.loader import WebLoader

import yarl

class NoArrayEnum(Init):
    def schemas(self, ctx: "Init.Context") -> "Init.Context":
        for n,s in ctx.schemas.items():
            if s.type == "array" and s.enum != None:
                s.enum = None
        return ctx

def test_jira():
    url = "https://dac-static.atlassian.com/cloud/jira/platform/swagger-v3.v3.json"
    loader = WebLoader(yarl.URL(url))
    with pytest.raises(aiopenapi3.errors.DiscriminatorWarning):
        api = OpenAPI.load_sync(url, loader=loader, plugins=[NoArrayEnum()])

[caseyoneill]

Thanks for the suggestion. I'm hoping that one library will "just work" like the Swagger Java library. FYI, I can reproduce this error in 1.7.0 too. I'm attaching a sample file.

recursion.json

As far as I can tell, the official Swagger parser does not chase down resolving component schema $ref items if they are attached as properties. The main parser loads a Component Processor. And then once a component property has a $ref it loads a Schema Processor. The Schema Processor only processes a $ref if it is an external reference.

[commonism]

You are basically asking for a client which accepts invalid description documents.

ms/kiota:

docker run -v ${PWD}:/app/output mcr.microsoft.com/openapi/kiota \
generate --language python --openapi https://dac-static.atlassian.com/cloud/jira/platform/swagger-v3.v3.json

Number of warnings/errors:

wc -l .kiota.log 
277 .kiota.log

Even if you got a library which "just works" as in "it can parse the description document", there may be differences in the protocol described by the description document, the protocol spoken by the service and the generated client protocol implementation.

I favor a strict client providing the facilities to address inconsistencies.