koxudaxi / datamodel-code-generator

Pydantic model and dataclasses.dataclass generator for easy conversion of JSON, OpenAPI, JSON Schema, and YAML data sources.
https://koxudaxi.github.io/datamodel-code-generator/
MIT License
2.44k stars 281 forks source link

patch pydantic v1.10.9 conflict with py3.12 #2014

Closed luca-knaack-webcom closed 6 days ago

luca-knaack-webcom commented 1 week ago

There are errors in pipelines that run the testing configuration python=3.12 and pydantic=1.10.9. The error is something like:

| ________________ ERROR collecting tests/parser/test_openapi.py _________________
| tests/parser/test_openapi.py:10: in <module>
|     from datamodel_code_generator import OpenAPIScope, PythonVersion
| datamodel_code_generator/__init__.py:35: in <module>
|     from datamodel_code_generator.parser.base import Parser
| datamodel_code_generator/parser/base.py:37: in <module>
|     from datamodel_code_generator.model import pydantic as pydantic_model
| datamodel_code_generator/model/__init__.py:5: in <module>
|     from ..types import DataTypeManager as DataTypeManagerABC
| datamodel_code_generator/types.py:240: in <module>
|     class DataType(_BaseModel):
| .venv/lib/python3.12/site-packages/pydantic/main.py:286: in __new__
|     cls.__try_update_forward_refs__()
| .venv/lib/python3.12/site-packages/pydantic/main.py:808: in __try_update_forward_refs__
|     update_model_forward_refs(cls, cls.__fields__.values(), cls.__config__.json_encoders, localns, (NameError,))
| .venv/lib/python3.12/site-packages/pydantic/typing.py:554: in update_model_forward_refs
|     update_field_forward_refs(f, globalns=globalns, localns=localns)
| .venv/lib/python3.12/site-packages/pydantic/typing.py:520: in update_field_forward_refs
|     field.type_ = evaluate_forwardref(field.type_, globalns, localns or None)
| .venv/lib/python3.12/site-packages/pydantic/typing.py:66: in evaluate_forwardref
|     return cast(Any, type_)._evaluate(globalns, localns, set())
| E   TypeError: ForwardRef._evaluate() missing 1 required keyword-only argument: 'recursive_guard'

and results in a massive amount of errors like:

ERROR tests/test_format.py - TypeError: ForwardRef._evaluate() missing 1 required keyword-only argument: 'recursive_guard'

To patch the missing recursive_guard keyword argument, I created a pydantic_patch file that gets imported early in datamodel_code_generator.__init__.py and that patches the underlying evaluate_forwardref function.

This PR is should fix most of the broken tests in the pipelines for pull requests #2013, #2009 and #2008.

codecov[bot] commented 1 week ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Please upload report for BASE (main@28be37d). Learn more about missing BASE report.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #2014 +/- ## ======================================= Coverage ? 98.81% ======================================= Files ? 38 Lines ? 4215 Branches ? 981 ======================================= Hits ? 4165 Misses ? 31 Partials ? 19 ``` | [Flag](https://app.codecov.io/gh/koxudaxi/datamodel-code-generator/pull/2014/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Koudai+Aono) | Coverage Δ | | |---|---|---| | [unittests](https://app.codecov.io/gh/koxudaxi/datamodel-code-generator/pull/2014/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Koudai+Aono) | `98.48% <100.00%> (?)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Koudai+Aono#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

luca-knaack-webcom commented 1 week ago

While there is still one failing pipeline, this is the godspeed-benchmark, that isn't able to run on any PR's pipeline.

codspeed-hq[bot] commented 1 week ago

CodSpeed Performance Report

Merging #2014 will not alter performance

Comparing luca-knaack-webcom:fix/pydantic-1-10-9-with-py-3-12 (611d070) with main (28be37d)

Summary

✅ 29 untouched benchmarks

luca-knaack-webcom commented 1 week ago

@koxudaxi, I had to change the codspeed-benchmark action to always install the dependencies, as the cached virtual environment seems to lack the pytest-xdist package. Once the cache is refreshed, the commented out line could be reintroduced.

koxudaxi commented 6 days ago

@cclauss Thank you for spending so much time on this difficult issue. I was commenting on the code, but I am concerned about the support for other versions, but I don't think it is a problem now, so I will merge it. Thank you very much.

koxudaxi commented 6 days ago

For performance reasons, you may also want to include a version check on the pydantic side.