swagger-api / swagger-parser

Swagger Spec to Java POJOs
http://swagger.io
Apache License 2.0
785 stars 531 forks source link

Allow user to override the Code Point Limit required by SnakeYaml #1872

Closed AmateurECE closed 1 year ago

AmateurECE commented 1 year ago

With this patch, a user may set the system property 'maxYamlCodePoints' in order to override the default 3MiB limit configured in the org.yaml.snakeyaml package by default. This limit was implemented to prevent certain Denial-of-Service (DOS) attacks, but users should be given the opportunity to override this value for valid configurations which exceed the limit, such as the Redfish OpenAPI specification (developed by DMTF), which weighs in at 4.9MiB.

This patch was tested to work with openapi-generator-cli v6.3.0.

This PR closes #1871

Signed-off-by: Ethan D. Twardy ethan.twardy@gmail.com

AmateurECE commented 1 year ago

I believe this PR also fixes #1857.

frantuma commented 1 year ago

Thanks!

ShubhamShekhar1996 commented 1 year ago

Doesn't work for me though! Updated the dependency and Tried below command:- mvn clean install -DmaxYamlCodePoints="999999999" Still getting same error:- Caused by: org.yaml.snakeyaml.error.YAMLException: The incoming YAML document exceeds the limit: 3145728 code points

Can you guide me where I may be wrong?

AmateurECE commented 1 year ago

Hmm...which tool are you using, and can you provide the command line that generates the exception? I was using openapi-generator-cli. It's possible there's another code path that results in construction of a ScannerImpl without checking the system properties.

ShubhamShekhar1996 commented 1 year ago

I am using Git bash to run my builds .... I think its not taking the "maxYamlCodePoints"

AmateurECE commented 1 year ago

I'm sorry, I meant which tool is responsible for the error? Are you using swagger-parser-cli to do validation, or are you using openapi-generator-cli to generate client/server code for an API? Can you provide the command that you are running in Git Bash?

ShubhamShekhar1996 commented 1 year ago

I'm sorry, I meant which tool is responsible for the error? Are you using swagger-parser-cli to do validation, or are you using openapi-generator-cli to generate client/server code for an API? Can you provide the command that you are running in Git Bash?

I am using "swagger-parser-cli". Below is the command:-

mvn clean install -DmaxYamlCodePoints="99"

AmateurECE commented 1 year ago

Ah, I see. This system property is meant to be interpreted at runtime, not at build time. So, if you're running swagger-parser-cli, I suggest setting the system property in that invocation, like so:

$ java -DmaxYamlCodePoints=999999 -jar ~/.m2/repository/[...]/swagger-parser-cli-<version>.jar <other arguments>
ShubhamShekhar1996 commented 1 year ago

Tried below command :- java -DmaxYamlCodePoints=999999 -jar swagger-parser-v3-2.1.10.jar But it gives below error though :- no main manifest attribute, in swagger-parser-v3-2.1.10.jar

I searched but I could not find swagger-parser-cli-.jar in my local m2 repo.Only thing I could find was .....m2\repository\io\swagger\parser\v3\swagger-parser-v3\2.1.10\wagger-parser-v3-2.1.10.jar

Below is the dependency I am using in my pom.xml:-

io.swagger.parser.v3 swagger-parser 2.1.11

Also is there any way to use it during build (mvn install)?

ShubhamShekhar1996 commented 1 year ago

Any suggestions?

elmehi commented 1 year ago

@AmateurECE , i'm running into difficulty with a command of the form: java -DmaxYamlCodePoints=999999 -jar ./generator/openapi-generator-cli-6.4.0.jar <other arguments>

error: Caused by: org.yaml.snakeyaml.error.YAMLException: The incoming YAML document exceeds the limit: 3145728 code points.

any ideas what's wrong?

AmateurECE commented 1 year ago

I believe openapi-generator is still using version 2.1.6 of swagger-parser. This PR was only merged as recently as 2.1.11. So I expect that when they upgrade the version in their Maven configuration, this issue will be resolved. You can build it from source and manually change the version of swagger-parser as a workaround.

AmateurECE commented 1 year ago

Any suggestions?

@ShubhamShekhar1996 The system property just needs to be set whenever swagger-parser is run, so as long as you can set the system property while you're running the swagger maven plugin, it should work for you. I expect your command above did not work because you don't have the library available in your local maven repository. You will have to run mvn install.

AmateurECE commented 1 year ago

@Samarth Gupta, the example you provided appears to be setting the maxYamlCodePoints value as an environment variable, but this mechanism is triggered by a JVM system property. Environment variables cannot be used to change this value. See https://stackoverflow.com/a/7055010

On Tue, Jul 25, 2023 at 6:35 AM Samarth Gupta @.***> wrote:

I am still getting limit exceed error even when I set the required env variable. Below is sample

ParseOptions options = new ParseOptions(); options.setResolve(true); options.setResolveFully(true); System.out.println(System.getenv("maxYamlCodePoints")); String openApiSpec = FileUtils.readFileToString(new File("openapi.yaml")); OpenAPI openAPI = new OpenAPIParser().readContents(openApiSpec, emptyList(), options).getOpenAPI();

error

Caused by: org.yaml.snakeyaml.error.YAMLException: The incoming YAML document exceeds the limit: 3145728 code points.

have set maxYamlCodePoints to 9999999999

— Reply to this email directly, view it on GitHub https://github.com/swagger-api/swagger-parser/pull/1872#issuecomment-1649206821, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEU5Q3HPLNITKLRCHNQZHQTXR5SLFANCNFSM6AAAAAAT3CCQLA . You are receiving this because you were mentioned.Message ID: @.***>

svghadi commented 8 months ago

Java system properties can be set using _JAVA_OPTIONS env var. Below worked for me

export _JAVA_OPTIONS=-DmaxYamlCodePoints=99999999