search

swagger-api / swagger-parser

Swagger Spec to Java POJOs
http://swagger.io
Apache License 2.0
786 stars 531 forks source link

Unable to load an OpenAPI3 spec from the classpath #637

Closed InfoSec812 closed 6 years ago

InfoSec812 commented 6 years ago

In my Vert.x 3.5.0 project, I am trying to use an OpenAPI3 specification stored in src/main/resources/adjective.yaml. The problem is that the URL for the file should be classpath:adjective.yaml, but the code in the following line:

https://github.com/swagger-api/swagger-parser/blob/v2.0.0-rc1/modules/swagger-parser-v3/src/main/java/io/swagger/parser/v3/OpenAPIV3Parser.java#L112

This code assumes the URL MUST be either http or file. This will not allow from loading the spec from the classpath.

gracekarina commented 6 years ago

@InfoSec812 Hi, what error do you get?

InfoSec812 commented 6 years ago

NullPointerException...

The code runs fine when the spec file is present on the local filesystem (e.g. Running in my IDE or from Maven), but once I package the whole thing up as a FatJar which includes the spec file, the parser is unable to read the file from inside of the fatjar's classpath.

Example code available at: https://github.com/rhoar-shootout/rhoar-vertx

Thanks!

gracekarina commented 6 years ago

@InfoSec812 thanks!

gracekarina commented 6 years ago

Hi @InfoSec812 , I haven't been able to run the project, it has an exception. Can you please send me the steps to reproduce the issue? thanks.


Running com.redhat.labs.insult.MainVerticleSpec
INFO 26/01/18 03:18 PM: liquibase: Successfully acquired change log lock
INFO 26/01/18 03:18 PM: liquibase: Successfully acquired change log lock
INFO 26/01/18 03:18 PM: liquibase: Reading from PUBLIC.DATABASECHANGELOG
INFO 26/01/18 03:18 PM: liquibase: Reading from PUBLIC.DATABASECHANGELOG
INFO 26/01/18 03:18 PM: liquibase: Successfully released change log lock
INFO 26/01/18 03:18 PM: liquibase: Reading from PUBLIC.DATABASECHANGELOG
INFO 26/01/18 03:18 PM: liquibase: Successfully released change log lock
[MLog-Init-Reporter] INFO com.mchange.v2.log.MLog - MLog clients using slf4j logging.
[vert.x-eventloop-thread-1] INFO com.mchange.v2.c3p0.C3P0Registry - Initializing c3p0-0.9.5.2 [built 08-December-2015 22:06:04 -0800; debug? true; trace: 10]
ene 26, 2018 3:18:45 PM io.vertx.core.impl.TaskQueue
GRAVE: Caught unexpected Throwable
java.lang.IllegalStateException: Result is already complete: failed
    at io.vertx.core.impl.FutureImpl.fail(FutureImpl.java:103)
    at io.vertx.core.impl.ContextImpl.lambda$executeBlocking$1(ContextImpl.java:284)
    at io.vertx.core.impl.TaskQueue.run(TaskQueue.java:80)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:748)

ene 26, 2018 3:18:45 PM io.vertx.core.impl.TaskQueue
GRAVE: Caught unexpected Throwable
java.lang.IllegalStateException: Result is already complete: failed
    at io.vertx.core.impl.FutureImpl.fail(FutureImpl.java:103)
    at io.vertx.core.impl.ContextImpl.lambda$executeBlocking$1(ContextImpl.java:284)
    at io.vertx.core.impl.TaskQueue.run(TaskQueue.java:80)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:748)

Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 22.107 sec <<< FAILURE!
com.redhat.labs.insult.MainVerticleSpec  Time elapsed: 0 sec  <<< FAILURE!
Async conditions timed out after 20,00 seconds; 2 out of 3 evaluate blocks did not complete in time
    at spock.util.concurrent.AsyncConditions.await(AsyncConditions.java:144)
    at com.redhat.labs.insult.MainVerticleSpec.setupSpec(MainVerticleSpec.groovy:53)

Results :

Failed tests:
  com.redhat.labs.insult.MainVerticleSpec

Tests run: 1, Failures: 1, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] microservices ...................................... SUCCESS [  0.573 s]
[INFO] adjective .......................................... SUCCESS [ 20.009 s]
[INFO] noun ............................................... SUCCESS [ 12.359 s]
[INFO] insult ............................................. FAILURE [ 26.471 s]
[INFO] ui ................................................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 59.743 s
[INFO] Finished at: 2018-01-26T15:19:01-05:00
[INFO] Final Memory: 66M/463M
[INFO] ---------------------------------------------
InfoSec812 commented 6 years ago

That is the exact exception caused by not being able to read the spec file. Let me tag a version and I will get back to you.

Deven Phillips Senior Consulting Engineer Red Hat, Open Innovation Labs

On Jan 26, 2018 3:32 PM, "Grace Karina Gonzalez Diaz" < notifications@github.com> wrote:

Hi @InfoSec812 https://github.com/infosec812 , I haven't been able to run the project, it has an exception. Can you please send me the steps to reproduce the issue? thanks.

Running com.redhat.labs.insult.MainVerticleSpec INFO 26/01/18 03:18 PM: liquibase: Successfully acquired change log lock INFO 26/01/18 03:18 PM: liquibase: Successfully acquired change log lock INFO 26/01/18 03:18 PM: liquibase: Reading from PUBLIC.DATABASECHANGELOG INFO 26/01/18 03:18 PM: liquibase: Reading from PUBLIC.DATABASECHANGELOG INFO 26/01/18 03:18 PM: liquibase: Successfully released change log lock INFO 26/01/18 03:18 PM: liquibase: Reading from PUBLIC.DATABASECHANGELOG INFO 26/01/18 03:18 PM: liquibase: Successfully released change log lock [MLog-Init-Reporter] INFO com.mchange.v2.log.MLog - MLog clients using slf4j logging. [vert.x-eventloop-thread-1] INFO com.mchange.v2.c3p0.C3P0Registry - Initializing c3p0-0.9.5.2 [built 08-December-2015 22:06:04 -0800; debug? true; trace: 10] ene 26, 2018 3:18:45 PM io.vertx.core.impl.TaskQueue GRAVE: Caught unexpected Throwable java.lang.IllegalStateException: Result is already complete: failed at io.vertx.core.impl.FutureImpl.fail(FutureImpl.java:103) at io.vertx.core.impl.ContextImpl.lambda$executeBlocking$1(ContextImpl.java:284) at io.vertx.core.impl.TaskQueue.run(TaskQueue.java:80) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748)

ene 26, 2018 3:18:45 PM io.vertx.core.impl.TaskQueue GRAVE: Caught unexpected Throwable java.lang.IllegalStateException: Result is already complete: failed at io.vertx.core.impl.FutureImpl.fail(FutureImpl.java:103) at io.vertx.core.impl.ContextImpl.lambda$executeBlocking$1(ContextImpl.java:284) at io.vertx.core.impl.TaskQueue.run(TaskQueue.java:80) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748)

Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 22.107 sec <<< FAILURE! com.redhat.labs.insult.MainVerticleSpec Time elapsed: 0 sec <<< FAILURE! Async conditions timed out after 20,00 seconds; 2 out of 3 evaluate blocks did not complete in time at spock.util.concurrent.AsyncConditions.await(AsyncConditions.java:144) at com.redhat.labs.insult.MainVerticleSpec.setupSpec(MainVerticleSpec.groovy:53)

Results :

Failed tests: com.redhat.labs.insult.MainVerticleSpec

Tests run: 1, Failures: 1, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] microservices ...................................... SUCCESS [ 0.573 s] [INFO] adjective .......................................... SUCCESS [ 20.009 s] [INFO] noun ............................................... SUCCESS [ 12.359 s] [INFO] insult ............................................. FAILURE [ 26.471 s] [INFO] ui ................................................. SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 59.743 s [INFO] Finished at: 2018-01-26T15:19:01-05:00 [INFO] Final Memory: 66M/463M [INFO] ---------------------------------------------

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/swagger-api/swagger-parser/issues/637#issuecomment-360896688, or mute the thread https://github.com/notifications/unsubscribe-auth/AAGFa-zXNv1mKFUI_dDSqQtLiYa1HdEpks5tOjZpgaJpZM4RpEx8 .

InfoSec812 commented 6 years ago

So, there is no solution to the error other than making the swagger-parser handle Spec files from the classpath... The reason this build fails is that the BDD tests written in SpockFramework are trying to launch the application in a semi-integrated form and unable to load the spec from the classpath.. If you wish to get past the error you can do:

mvn clean package -pl adjective,noun,insult -DskipTests

You could then run one of the services using:

java -jar adjective/target/adjective-1.0.0-SNAPSHOT.jar

But you will get that same error because it cannot load the spec YAML file from the classpath.

InfoSec812 commented 6 years ago

Woohoo!!! I just pulled down the 2.0 branch and it works perfectly! Yay!

NVM, I spoke too soon... I still had the relative path in there.

InfoSec812 commented 6 years ago

I tried using the following path syntaxes:

./spec.yaml
/spec.yaml
spec.yaml
classpath:spec.yaml
classpath:/spec.yaml
classpath:./spec.yaml

But none of them worked.

gracekarina commented 6 years ago

Can you decompress the jar, and see where is the yaml being saved?

InfoSec812 commented 6 years ago

The file is in the root of the JAR file.

$ jar -tvf insult/target/insult-1.0.0-SNAPSHOT.jar | grep yaml
  2393 Mon Jan 29 14:20:28 EST 2018 insult.yaml
gracekarina commented 6 years ago

wait I thought we where looking at adjetive.yaml. btw I'm trying to run it but it says there is not manifest in the jar. I'm also trying to build the project from intelliJ so I can debug and see exactly where the parser is being called, I see it's in a OpenAPIV3Factory.

gracekarina commented 6 years ago

Do you Know where I should add the spock library? I added the groovy and those errors were fixed but the spock.lang.Specification it's still there. I looked in maven but there is not spock library, What am I doing wrong?

InfoSec812 commented 6 years ago

Hrrmm... It's in the POM file... Not sure why it wouldn't be working for you. I too use IntelliJ. As for the YAML files, there are 3, one for each service: adjective, noun, and insult.

InfoSec812 commented 6 years ago

Oh! And to compile an executable jar:

mvn clean package vertx:package -pl adjective,noun,insult -DskipTests

Sorry, I forgot about the vertx:package step

gracekarina commented 6 years ago

yeah I see the dependency in the pom but it's not taking it.

InfoSec812 commented 6 years ago

Oh wait... I just looked back through your error and the Spock test IS running, but it fails because the Spec file cannot be read from the classpath.

gracekarina commented 6 years ago

So I manage to fix the issues with intelliJ, just had to set the target generated tests to a test folder in the project structure.

And doing a remote debbuging seems like when they're trying to set the File (before the calling of parser) the path it's not found:

Future{cause=Wrong specification url/path: file:/Users/gracekarina/repo/rhoar-vertx/adjective/target/adjective-1.0.0-SNAPSHOT.jar!/adjective.yaml}

This is the route it is being given to the parser: spec.getAbsolutePath() /Users/gracekarina/repo/rhoar-vertx/file:/Users/gracekarina/repo/rhoar-vertx/adjective/target/adjective-1.0.0-SNAPSHOT.jar!/adjective.yaml

Interface: OpenAPI3RouterFactory


static void createRouterFactoryFromFile(Vertx vertx, String filename, Handler<AsyncResult<OpenAPI3RouterFactory>>
    handler) {
    vertx.executeBlocking((Future<OpenAPI3RouterFactory> future) -> {
      File spec = new File(filename);
      if (!spec.exists()) 
        future.fail(RouterFactoryException.createSpecNotExistsException(filename));
      SwaggerParseResult swaggerParseResult = new OpenAPIV3Parser().readLocation(spec.getAbsolutePath(), null, null);
      if (swaggerParseResult.getMessages().isEmpty()) future.complete(new OpenAPI3RouterFactoryImpl(vertx, swaggerParseResult.getOpenAPI()));
      else {
       future.fail(RouterFactoryException.createSpecInvalidException(StringUtils.join(swaggerParseResult.getMessages(),", ")));
      }
    }, handler);
  }
InfoSec812 commented 6 years ago

I'll have a look and get back to you.

slinkydeveloper commented 6 years ago

@gracekarina I'm trying to use the method that loads from url createRouterFactoryFromURL and It doesn't work. Giving a deeper look it seems that parser doesn't support the jar protocol. For example, If I pass the below url:

jar:file:/home/francesco/IdeaProjects/cp-bug-oas3-vertx/target/cp-bug-oas3-vertx-1.0-SNAPSHOT-fat.jar!/swagger.json

While it's parsed by OpenAPIV3Parser.readWithInfo() it checks if starts or with file or with http

slinkydeveloper commented 6 years ago

@gracekarina after playing a bit, I've decided to create one single method create() that accepts relative/absolute paths and file/http urls and leave the work to parser: https://github.com/vert-x3/vertx-web/blob/contracts_fix/vertx-web-api-contract/src/main/java/io/vertx/ext/web/api/contract/openapi3/OpenAPI3RouterFactory.java#L93

But the jar protocol still doesn't work

gracekarina commented 6 years ago

@slinkydeveloper @InfoSec812 can you please update the project so I can reproduce it locally with the new create() method? thanks.

InfoSec812 commented 6 years ago

@gracekarina Unfortunately, this is work on a pre-release branch which will not be accessible for you to test unless you want to download and build vertx-web yourself. Once @slinkydeveloper and I can get it working, we will update this issue.

gracekarina commented 6 years ago

Hi I was able to reproduce it with a small sample. Working on it!

gracekarina commented 6 years ago

@slinkydeveloper about jar:file:/home/francesco/IdeaProjects/cp-bug-oas3-vertx/target/cp-bug-oas3-vertx-1.0-SNAPSHOT-fat.jar!/swagger.json is there a reason why you can't send the url without being a resource? just send ´/adjective.yaml´, because I've tested, and parser solves the classpath even with the remote fatJar but only when I put the simple url. If I use the getClass.getResource("/adjective.yaml").getFile(); this just sends to parser the absoluteURL, and parser wont find the file in the absolute location, just asking to make sure it's a bug, or maybe it can be used in another way. thanks!

gracekarina commented 6 years ago

@slinkydeveloper , @InfoSec812 I did this(just passed the string of the simple path adjective.yaml) in the MainVerticle.java class in adjective module and got an OpenAPI resolved and a clean build (I builded locally the changes in vertx):


private Future<OpenAPI3RouterFactory> provisionRouter(Void v) {
        service = new AdjectiveServiceImpl(vertx);
        new ServiceBinder(vertx).setAddress("adjective.service").register(AdjectiveService.class, service);
        Future<OpenAPI3RouterFactory> future = Future.future();
        CircuitBreaker breaker = CircuitBreaker.create("openApi", vertx, new CircuitBreakerOptions()
                .setMaxFailures(5) // number of failure before opening the circuit
                .setTimeout(200000) // consider a failure if the operation does not succeed in time
                .setFallbackOnFailure(false) // do we call the fallback on failure
                .setResetTimeout(1000000));

        breaker.<OpenAPI3RouterFactory>execute(f -> OpenAPI3RouterFactory.createRouterFactoryFromFile(
                vertx,
                "/adjective.yaml",
                f.completer())).setHandler(future.completer());

        return future;
    }
InfoSec812 commented 6 years ago

@gracekarina I believe the (as yet unreleased) changes which @slinkydeveloper made for the 3.5.1 version of Vert.x are working perfectly and we can close this issue.

icancode007 commented 2 years ago

A couple years since this issue, but currently facing something similar within vertx attempting to read the file from fatJar packaging where dir structure looks like the following

>io
  >v3
    >parser
     >core
     .
     .
     .
>com
   >attlaisan
   .
   .
   .     
>SwaggerFallback.json

The below code reads the file fine locally 

 builder = OpenApiInteractionValidator
          .createForSpecificationUrl("/SwaggerFallback.json")
          .build();

but in a higher env when its packaged up fails silently after that call same call, well not completely it shows this bit in the stacktrace 

error":{"name":"java.util.ServiceConfigurationError","message":"io.swagger.v3.parser.core.extensions.SwaggerParserExtension: io.swagger.v3.parser.converter.SwaggerConverter Unable to get public no-arg constructor"

sadly can't share the code as it pertains to a private repo, but if theres any advise of how I should be reading the file. would greatly appreciate it. @gracekarina

InfoSec812 commented 2 years ago

There was a discussion on the Vert.x Discord about this and it has nothing to do with swagger-parser. You need to set a System Property as shown here: https://github.com/InfoSec812/budjet-vertx-hibernate-reactive-openapi/blob/main/modules/api/src/main/java/com/zanclus/BudjetLauncher.java#L6