search

bundesAPI / smard-api

https://smard.api.bund.dev
54 stars 8 forks source link

How does the code generation work? #17

Open Armagetron opened 1 year ago

Armagetron commented 1 year ago

I tried to update the API specification for the projected energy generation. However I couldn't get the generator to work. For local use, I should specify the template, but I couldn't find the template files anywhere. The generator itself later fails, see output below. Is there a documentation available on how to run the generator for this project?

java -jar openapi-generator-cli-6.2.1.jar generate -g python -i generator_config.yaml
[main] ERROR i.s.parser.SwaggerCompatConverter - failed to read resource listing
com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'templateDir': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
 at [Source: (String)"templateDir: ./local/deutschland_templates
additionalProperties:
  packageName: "smard"
  infoName: "BundesAPI"
  infoEmail: "kontakt@bund.dev"
  packageVersion: 0.1.0
  packageUrl: "https://github.com/bundesAPI/smard-api"
  namespace: "deutschland"
  docLanguage: "de"
gitHost: "github.com"
gitUserId: "bundesAPI"
gitRepoId: "smard"
files:
  pyproject.mustache:
    destinationFilename: pyproject.toml
    templateType: SupportingFiles
  requirements.txt: {}
  create_doc.mustache:
    destinationFi"[truncated 262 chars]; line: 1, column: 12]
        at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:2391)
        at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:745)
        at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._reportInvalidToken(ReaderBasedJsonParser.java:2961)
        at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._reportInvalidToken(ReaderBasedJsonParser.java:2939)
        at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._matchToken(ReaderBasedJsonParser.java:2713)
        at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._matchTrue(ReaderBasedJsonParser.java:2667)
        at com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:767)
        at com.fasterxml.jackson.databind.ObjectMapper._readTreeAndClose(ObjectMapper.java:4703)
        at com.fasterxml.jackson.databind.ObjectMapper.readTree(ObjectMapper.java:3076)
        at io.swagger.parser.SwaggerCompatConverter.readResourceListing(SwaggerCompatConverter.java:209)
        at io.swagger.parser.SwaggerCompatConverter.read(SwaggerCompatConverter.java:122)
        at io.swagger.parser.SwaggerCompatConverter.readWithInfo(SwaggerCompatConverter.java:93)
        at io.swagger.parser.SwaggerParser.readWithInfo(SwaggerParser.java:45)
        at io.swagger.v3.parser.converter.SwaggerConverter.readLocation(SwaggerConverter.java:83)
        at io.swagger.parser.OpenAPIParser.readLocation(OpenAPIParser.java:16)
        at org.openapitools.codegen.config.CodegenConfigurator.toContext(CodegenConfigurator.java:573)
        at org.openapitools.codegen.config.CodegenConfigurator.toClientOptInput(CodegenConfigurator.java:631)
        at org.openapitools.codegen.cmd.Generate.execute(Generate.java:457)
        at org.openapitools.codegen.cmd.OpenApiGeneratorCommand.run(OpenApiGeneratorCommand.java:32)
        at org.openapitools.codegen.OpenAPIGenerator.main(OpenAPIGenerator.java:66)
[main] ERROR i.s.parser.SwaggerCompatConverter - failed to read resource listing
com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'templateDir': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
 at [Source: (String)"templateDir: ./local/deutschland_templates
additionalProperties:
  packageName: "smard"
  infoName: "BundesAPI"
  infoEmail: "kontakt@bund.dev"
  packageVersion: 0.1.0
  packageUrl: "https://github.com/bundesAPI/smard-api"
  namespace: "deutschland"
  docLanguage: "de"
gitHost: "github.com"
gitUserId: "bundesAPI"
gitRepoId: "smard"
files:
  pyproject.mustache:
    destinationFilename: pyproject.toml
    templateType: SupportingFiles
  requirements.txt: {}
  create_doc.mustache:
    destinationFi"[truncated 262 chars]; line: 1, column: 12]
        at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:2391)
        at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:745)
        at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._reportInvalidToken(ReaderBasedJsonParser.java:2961)
        at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._reportInvalidToken(ReaderBasedJsonParser.java:2939)
        at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._matchToken(ReaderBasedJsonParser.java:2713)
        at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._matchTrue(ReaderBasedJsonParser.java:2667)
        at com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:767)
        at com.fasterxml.jackson.databind.ObjectMapper._readTreeAndClose(ObjectMapper.java:4703)
        at com.fasterxml.jackson.databind.ObjectMapper.readTree(ObjectMapper.java:3076)
        at io.swagger.parser.SwaggerCompatConverter.readResourceListing(SwaggerCompatConverter.java:209)
        at io.swagger.parser.SwaggerCompatConverter.read(SwaggerCompatConverter.java:122)
        at io.swagger.parser.SwaggerCompatConverter.readWithInfo(SwaggerCompatConverter.java:93)
        at io.swagger.parser.SwaggerParser.readWithInfo(SwaggerParser.java:45)
        at io.swagger.v3.parser.converter.SwaggerConverter.readLocation(SwaggerConverter.java:83)
        at io.swagger.parser.OpenAPIParser.readLocation(OpenAPIParser.java:16)
        at org.openapitools.codegen.config.CodegenConfigurator.toContext(CodegenConfigurator.java:573)
        at org.openapitools.codegen.config.CodegenConfigurator.toClientOptInput(CodegenConfigurator.java:631)
        at org.openapitools.codegen.cmd.Generate.execute(Generate.java:457)
        at org.openapitools.codegen.cmd.OpenApiGeneratorCommand.run(OpenApiGeneratorCommand.java:32)
        at org.openapitools.codegen.OpenAPIGenerator.main(OpenAPIGenerator.java:66)
Exception in thread "main" org.openapitools.codegen.SpecValidationException: There were issues with the specification. The option can be disabled via validateSpec (Maven/Gradle) or --skip-validate-spec (CLI).
 | Error count: 9, Warning count: 0
Errors: 
        -attribute paths is missing
        -attribute additionalProperties is unexpected
        -attribute swagger is missing
        -attribute gitRepoId is unexpected
        -attribute templateDir is unexpected
        -attribute gitUserId is unexpected
        -attribute files is unexpected
        -attribute gitHost is unexpected
        -attribute info is missing

        at org.openapitools.codegen.config.CodegenConfigurator.toContext(CodegenConfigurator.java:604)
        at org.openapitools.codegen.config.CodegenConfigurator.toClientOptInput(CodegenConfigurator.java:631)
        at org.openapitools.codegen.cmd.Generate.execute(Generate.java:457)
        at org.openapitools.codegen.cmd.OpenApiGeneratorCommand.run(OpenApiGeneratorCommand.java:32)
        at org.openapitools.codegen.OpenAPIGenerator.main(OpenAPIGenerator.java:66)
wirthual commented 1 year ago

Hi,

The templates live in the generator action repo here:

https://github.com/wirthual/deutschland-generator-action/tree/main/deutschland_templates

You should be able to download them and set the path in the generator_config.yaml accordingly.

Also you need to execute this script after the generation to bring it in the right folder structure: https://github.com/wirthual/deutschland-generator-action/blob/main/rename_generated_code.py

Basically you can follow the steps in action.yaml to see what's going on.

Let me know if that works for you.

Unfortunately there are no instructions on how to do this now, so we would really appreciate if you could write down your steps so we can save others from you struggle.

Best, wirthual

Armagetron commented 1 year ago

I somehow missed your reply but now I have looked into it. The following how-to should cover all the steps. I didn't manage to build the client under the deutschland package. Do you have any tips on that?

How to build

This tutorial explains how to build the smard-api on your local machine. This tutorial assumes that you already have the smard-api repo cloned to your local machine.

OpenAPI generator

First, you need an OpenAPI generator installed. Installation is described on the project website. This tutorial assumes that the generator jar is present on your machine.

The version is important. You can find the corresponding version in the file action.yaml in the generator-tag. As of writing this, this is version 6.1.0. Make sure to download the corresponding version or at least the same minor version.

Templates

The generator needs templates. They reside in a different repository, called deutschland-generator-action. Clone this repository outside of the smard-api repository:

git clone https://github.com/wirthual/deutschland-generator-action.git

Now you need to adjust the templateDir in the generator_config.yaml file so that the generator knows where to find the templates.

templateDir: /path/to/deutschland-generator-action/deutschland_templates

Generating the python client

Now that you have the generator installed and the templates configured, you can generate the python client. Make sure that your working directory is the smard-api root. The output directory must be the python-client folder. Otherwise later steps will fail. Remove all old contest with

rm -rf python-client/*

The OpenAPI generator takes at least four arguments, the openapi configuration file, the language generator, the generator config and the output directory. Make sure to use the openapi.yaml file as the input-spec and the generator_config.yaml file as the config file. You can invoke the generator like this:

java -jar openapi-generator-cli.jar generate -c generator_config.yaml -i openapi.yaml -g python -o python-client

Finally, you need to adjust some path. Use the rename_generated_code.py script from the deutschland-generator-action repository.

python /path/to/deutschland-generator-action/rename_generated_code.py python-client

Installing

Now that you have successfully build the smard-api client, you can install it on your system. In the python-client folder run the python tool poetry command:

poetry dist

Now you should have a de_smard-0.1.0.tar.gz file in the python-client/dist folder. You can install it using pip:

pip install -U python-client/dist/de_smard-0.1.0.tar.gz
Armagetron commented 1 year ago

However you can do import smard instead of from deutschland import smard.