koxudaxi / datamodel-code-generator

Pydantic model and dataclasses.dataclass generator for easy conversion of JSON, OpenAPI, JSON Schema, and YAML data sources.
https://koxudaxi.github.io/datamodel-code-generator/
MIT License
2.65k stars 296 forks source link

Consider replacing `datamodel-code-generator/datamodel_code_generator/reference.py` with the referencing library #1820

Open leonardopsantos opened 8 months ago

leonardopsantos commented 8 months ago

Context: I've spent the last week writing a code generation script that converts JSON schemas into Python dataclasses and C++ classes; and today I stumbled into your wonderful project!

My schemas are all file-based, an in my parser I uses the python-jsonschema/referencing library with a custom fetcher to handle the relative file references:

import json

from pathlib import Path
from urllib.parse import urlparse

import referencing

# Could use an object that holds the base folder as a property, using this for simplicity
BASE_FOLDER = Path()

def retrieve(self, uri:str) -> referencing.Resource:
    parsed_uri = urlparse(uri)
    p = Path(parsed_uri.path).relative_to("/")
    with open(BASE_FOLDER / p) as f:
        return referencing.Resource.from_contents(json.load(f))

def do_parsing():

    # Add the first file
    with open("Foo.schema.json", "r") as fp:
        jmap = json.load(fp)
        resource = referencing.Resource.from_contents(jmap)
        # Initial registry
        registry = referencing.Registry(retrieve=retrieve)

    # Recursively parse the schema, when `$ref` is found:

    # The value of $ref goes here
    # lookup is great!!
    referenced_json_schema = registry.resolver().lookup(jmap["$ref"])

    # referenced_json_schema is a Python dict with the JSON data

    # Update the registry with the referenced schema's ID
    registry = referenced_json_schema @ registry

Looking at datamodel-code-generator/datamodel_code_generator/reference.py, I think there's a lot of overlap between that and referencing.

Check out the referencing docs.

Cheers!

koxudaxi commented 8 months ago

@leonardopsantos Thank you for sharing. I'll consider it when I have time after I close my other issues sorted out!

koxudaxi commented 8 months ago

I read a little code. He does a great job as a JSON Schmea expert. Given that I have maintained this tool for many years, I really should have developed a library for such a reference as a spin-off. But I guess I didn't realize that at the time. I'll keep that in mind in the future.