koxudaxi / datamodel-code-generator

Pydantic model and dataclasses.dataclass generator for easy conversion of JSON, OpenAPI, JSON Schema, and YAML data sources.
https://koxudaxi.github.io/datamodel-code-generator/
MIT License
2.71k stars 298 forks source link

'NO'/"NO" (string enum value) gets serialized to `False_` ? #1653

Open tommyjcarpenter opened 11 months ago

tommyjcarpenter commented 11 months ago

I have a really bizarre issue where the string 'NO' (ISO country code for Norway - we need this literal string to be an enum value), as part of an enum, is getting serlialized to False

we have an enum of ISO codes that includes the literal string 'NO' - I have tried this without quotes, and with both single and double

    MyModel:
      type: object
      additionalProperties: false
      required:
      - country_code
      properties:
        country_code:
            type: string
            example: US
            enum:
              - 'AE'
              - 'AR'
             ...
              - 'NL'
              - 'NO' <----------- have tried just NO, 'NO', and "NO" 
              - 'NZ'
             ...

using the generation flags:

    --strict-types str bool 
    --set-default-enum-member 
    --enum-field-as-literal one 
    --use-default  
    --strict-nullable 
    --collapse-root-models 
    --output-model-type pydantic_v2.BaseModel

we get the following output!

class CountryCode(Enum):
    AE = "AE"
    AR = "AR"
    ...
    NL = "NL"
    False_ = False  <------------- !!
    NZ = "NZ"
    ...

'NO' becomes False! Can you please let me know how to not do that?

Thanks much

tommyjcarpenter commented 11 months ago

@koxudaxi 🙏

koxudaxi commented 11 months ago

I just tried to your example. But, the result includes NO image

koxudaxi commented 11 months ago

@tommyjcarpenter I guess your complaint comes from pyyaml behavior. https://github.com/yaml/pyyaml/issues/376

I confirmed the problem when I removed the single quotes. image

tommyjcarpenter commented 11 months ago

@koxudaxi that's it - my original comment of trying quotes wasn't quite correct, because our openapi goes through a resolver, which spits out a fully resolved openapi that is missing the quotes - which then uses this package, and produces those results

thanks so much for your help.. i will need to dig into the resolver to see if I can fix this 😢

koxudaxi commented 11 months ago

Do you use pyyaml? datamodel-code-generator ignores the behavior for timestamp string by using the code. https://github.com/koxudaxi/datamodel-code-generator/blob/54800a14fa22655e2e9bce7f710a1c9c0d1c8327/datamodel_code_generator/util.py#L54-L56

I can see the URL(tag:yaml.org,2002:bool) for bool in the yaml document https://yaml.org/type/bool.html

By the way, thank you for your sponsoring me :smile:

tommyjcarpenter commented 11 months ago

@koxudaxi i asked my company if i could sponsor you with their money for more.. but no luck yet. you've been extremely helpful, my pleasure

we are using ruamel as a preprocessing step before the generator, and that is what is losing the quotes, which only affects this NO case. so, im looking into the flags for that to ...not do this

This issue is near the top of the list of dumbest things I've spent time on 😂

koxudaxi commented 11 months ago

@tommyjcarpenter

i asked my company if i could sponsor you with their money for more.. but no luck yet. you've been extremely helpful, my pleasure

I'm happy to hear the comment:smile_cat:

we are using ruamel as a preprocessing step before the generator,

Can you convert the yaml to json before passing yaml to datamldel-code-generator? If my understanding is correct, you can avoid the yaml issue.

tommyjcarpenter commented 11 months ago

@koxudaxi i tried this:


cat apis/openapi_resolved.yaml | yq e -o=json - > /tmp/as_json.json
datamodel-codegen --strict-types str bool --set-default-enum-member --enum-field-as-literal one --input-file-type openapi --input /tmp/as_json.json --output indexapi_types/pydantic_v2_types.py --use-default  --strict-nullable --collapse-root-models --output-model-type pydantic_v2.BaseModel

but that didnt seem to work, im double checking

koxudaxi commented 11 months ago

@tommyjcarpenter

@koxudaxi this is even a bigger problem because the JSON is double quoted in that output:

Sorry, I don't understand the problem. I can generate the enum from the json file :thinking: image

tommyjcarpenter commented 11 months ago

@koxudaxi sorry you are right - I was looking at the wrong output 🤦 converting to JSON works!

should this be noted somewhere? This was a nasty one to track down.

thanks much