search

grpc-ecosystem / grpc-gateway

gRPC to JSON proxy generator following the gRPC HTTP spec
https://grpc-ecosystem.github.io/grpc-gateway/
BSD 3-Clause "New" or "Revised" License
18.26k stars 2.24k forks source link

Feature Request: OpenAPI 3 support #441

Open c4milo opened 7 years ago

c4milo commented 7 years ago

https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.0.md

achew22 commented 7 years ago

@c4milo, Can you elaborate on what it is you're interested in that isn't available today?

c4milo commented 7 years ago

@achew22, feature wise, nothing. What's concerning to me right now is that https://github.com/swagger-api/swagger-js/tree/2.x is no longer active. Most development effort is going towards supporting OpenAPI v3.

zxy198717 commented 7 years ago

+1

johanbrandhorst commented 7 years ago

The OpenAPI 3.0 spec has added support for Authentication (JWT, Oath, HTTP Basic), which is something that is missing from 2.0. I think this would make it worthwhile to migrate to generating 3.0 compliant definitions.

jsw commented 7 years ago

Also oneof?

tamalsaha commented 6 years ago

I suppose https://github.com/go-openapi/spec3 will be needed first.

ivucica commented 6 years ago

@johanbrandhorst Strictly speaking, that's not something that makes v3 support a must and a requirement. Consider that OpenAPI v2 does support OAuth2 authentication, Basic authentication and API keys.

I don't see in that PR where OpenAPI v3 adds JWT, and I suppose you meant OAuth2 and not OATH (different thigns)?

What I see as a new thing in that PR you quoted is "HTTP Bearer" and "Multiple Flows", which both may be useful. (Even though Bearer will be usually used with OAuth2, anyway, there might be people out there not using it that way.)

@tamalsaha That would be the right way to go, but protoc-gen-swagger has OpenAPI v2 schema, so that's not a blocker either.

veqryn commented 6 years ago

OpenAPI v3.0.0 adds support for specifying multiple servers (hosts).

unship commented 6 years ago

oneof sematics is a pain point for us

jones77 commented 5 years ago

OpenAPI 3.0 adds support for nullable types, 2.0 doesn't support them: https://github.com/swagger-api/swagger-editor/issues/1364#issuecomment-309530250

johanbrandhorst commented 5 years ago

I'm pretty sure there's an extension for V2 (x-nullable?).

johanbrandhorst commented 5 years ago

https://github.com/grpc-ecosystem/grpc-gateway/issues/900 is a real issue that would be covered by this.

ivucica commented 5 years ago

FTR I will be very happy to review forks of the swagger generator, or elegant refactors of the swagger generator that allow for both openapiv2 and openapiv3.

misrab commented 5 years ago

+1

lukaskellerstein commented 5 years ago

+1

zachgersh commented 5 years ago

Hey everyone,

So we are actually doing this (at least I am going to try). I pushed the first bits of OPENAPI v3 support and attached the WIP PR to this issue for all to review.

I'd love everyone to go have a look at the proto and WIP if they can. I WANT YOUR FEEDBACK ON THIS. Would love to know what sort of fields are required missing from the next version of this generator.

ivucica commented 5 years ago

@zachgersh I still have the notification email for your comment marked as 'unread' in my inbox, this is exciting and I hope to take a look soon.

JugrajSripalJain commented 5 years ago

is this done, if not, do we have any ETA?

ivucica commented 5 years ago

I am not doing this myself and I have not seen activity on #1059. @JugrajSripalJain, you can review my comments there and see if they align with the way you'd like to implement this.

zachgersh commented 5 years ago

It's not done and no ETA as of yet. I put this down for a bit as other work jumped my queue. We do plan to come back to this some point in Q1 next year but I'd be happy to see someone else grab it if they have time now.

ambeur commented 4 years ago

hi, is there any news about this feature?

glebmachine commented 4 years ago

Looking forwards OAS3 support, could you share your current plans about?

johanbrandhorst commented 4 years ago

I think this is on ice again unfortunately. If someone would like to pick this up please let me know.

srenatus commented 4 years ago

ℹ️ Should we want to do this, here's an auto-generated protobuf model of OpenAPI v3 (and v2, for that matter): https://github.com/googleapis/gnostic/blob/master/openapiv3/OpenAPIv3.proto

krishna-birla commented 4 years ago

Coming here from https://github.com/googleapis/gnostic/issues/162

@srenatus https://github.com/googleapis/gnostic doesn't inter-convert between proto files and open API specs, it uses message buffers instead (binary serialized data). More over, the open API specs to proto files conversion does not exploit the newest oneOf feature as expected. I have posted some use cases in the issues there. And lastly, the better use case is to manually write proto files and then generate both gRPC and REST clients/servers from it.

It would be great to have it in grpc-gateway itself. Happy to contribute.

johanbrandhorst commented 4 years ago

We'd love to have openapi 3 support! Please know that it may be a lot of work. Please have a look at the v2 branch and you can probably use protoc-gen-openapiv2 as a starting point.

zwiedmann-isp commented 4 years ago

@johanbrandhorst @krishna-birla I'm willing to contribute where I can as well, tho my time is limited, and at present time I'm able to get away with v2 generation and conversion using https://github.com/getkin/kin-openapi/tree/master/openapi2conv . @johanbrandhorst if you able to provide any context on the gist of the work as you understand it that'd be helpful. At present time it's not entirely clear to me why we define openapiv2.proto files and what we would define a v3 variant to achieve. Of course I'm sure once I dig into the code that'll be clearer.

johanbrandhorst commented 4 years ago

Thanks! Frankly, I'm not sure what the need is either, I've been getting by with v2 wherever I've needed it. I'll be happy to help answer questions about the code but I'm no expert on openapi.

aeneasr commented 3 years ago

There are several significant changes to OAS3:

  1. Multi-server and server templating with the latter being particularly interesting for open source software which typically runs on different port/hosts everywhere. Can also be useful for multi-tenant services. OAS2 does not support this at all.
  2. OAS3 supports all HTTP security schemes as opposed to OAS2's limited scope. Missing from OAS2 is OpenID Connect and cookie auth which is particularly interesting for browser clients (e.g. SPA) that do not support gRPC.
  3. OAS3 supports polymorphism with oneOf, anyOf, and not all of which are not supported in OAS2. In OAS2 we only get allOf which limits the options we have for dealing with polymorph types in REST.
  4. Better support for request bodies, and the cookie parameter type which is not implemented in OAS2.
  5. OAS3 supports wildcard response codes such as 4xx where in OAS2 we need to define each error code individually, even if the output is the same (except for the status code).
  6. Supports links, useful to document e.g. pagination - useful - among others - for HATEOAS compatibility.

As I've been involved in go-swagger, I know that https://github.com/go-openapi/spec3 is probably not going to see involvement from the original author. He stated in an issue that he does not intend to work on this any more as it is too much work.

I know that this is a lot of work to implement, but given previous comments around what actually changed I thought it's a good idea to provide a short list of interesting changes. I think supporting OAS3 spec generation could help a lot of OSS projects in their API documentation and REST client generation!

achew22 commented 3 years ago

@aeneasr I don't think anyone is saying that the request is invalid or a bad idea. In reading the scrollback it sounds like the problem is that the marginal effort to adopt OpenAPI v3 is not worth the marginal effort for existing maintainers.

It sounds like the calculus might be different for you. How can I/we help you in the process of bringing OpenAPI v3 support into grpc-gateway?

krishna-birla commented 3 years ago

@achew22 I think the first few steps should include:

  1. Create a channel with maintainers and external contributors willing to ship OAS3 into grpc-gateway.
  2. Create a document suit for design and implementation.
  3. Create feature issues under OAS3 tag.

Once we are sound with the required changes, we can start assigning issues and creating PRs.

achew22 commented 3 years ago

@krishna-birla,

  1. Here is our slack channel if you'd like to get in there
  2. Sounds great. Please post the link to your design document in here when it's ready.
  3. I've created a tag for this called OpenAPI v3 it should be possible when filing a new issue to tag it with that.
krishna-birla commented 3 years ago

@krishna-birla,

  1. Here is our slack channel if you'd like to get in there

  2. Sounds great. Please post the link to your design document in here when it's ready.

  3. I've created a tag for this called OpenAPI v3 it should be possible when filing a new issue to tag it with that.

Getting krishnabirla16@gmail.com doesn’t have an account on this workspace. Can you add me please.

achew22 commented 3 years ago

It's just a link to the #grpc-gateway room inside the gopher slack. You can sign up here.

vtolstov commented 3 years ago

i'm read issue topic, what the problem now? incomplete spec or incomplete go representation of needed struct in spec? offtopic: does gnostic now supports all oneof/allof/anyof features from openapi v3?

krishna-birla commented 3 years ago

Looking at their latest examples, https://github.com/google/gnostic/tree/master/openapiv3, I think they do what we expect. But still not sure it is stable, at least not the last time I tried it (there was data loss in inter conversion).

anjmao commented 3 years ago

Hi, just to let you know that I started working on protoc-gen-openapiv3 which will support openapi v3. Will create WIP PR as soon as I add most important bits to template.go

DanielSolomon commented 2 years ago

Any update regarding this issue?

johanbrandhorst commented 2 years ago

To my knowledge, no one is actively working on this.

oxisto commented 2 years ago

Any update regarding this issue?

To share some insight, in our project http://github.com/clouditor/clouditor we are basically using a combination of grpc-gateway to generate the REST server as well as gnostic (> 0.6.0) to generate OpenAPI files and so far we had no problems with external clients.

You just have to make sure that your style for properties matches, gnostic seems to use camelCase, which also seems to be the default for grpc-gateway now, but this ca be configured: https://grpc-ecosystem.github.io/grpc-gateway/docs/mapping/customizing_your_gateway/#using-proto-names-in-json

pscott31 commented 2 years ago

I would love to see this feature and would be happy to contribute to it a bit if someone who knows what they are doing can get the ball rolling and point me at the right bits :)

johanbrandhorst commented 2 years ago

This is a big task. We've had a few users try to contribute this over the years. I think the only thing that will be successful is a from-scratch written generator with a minimal amount of features from OpenAPIv3 that can be extended by the community.

ianrose14 commented 2 years ago

FWIW my current approach is to use grpc-gateway to produce files in OpenAPI 2 format (via the standard -openapiv2_out option) and then convert those to OpenAPI 3 format via https://github.com/swagger-api/swagger-codegen:

java -jar /swagger/swagger-codegen-cli.jar generate -l openapi -i "$INFILE" -o "$TEMPDIR"

MikyChow commented 2 years ago

still waiting for this feature

sagikazarmark commented 2 years ago

Looking at this problem at work right now. I used gnostic to generate OpenAPI spec from proto. I'll be adding grpc-gateway soon. I'll report back how it works.

In the meantime, I found this: https://github.com/sv-tools/openapi

Bunch of other libs listed on this site: https://github.com/OAI/OpenAPI-Specification/blob/main/IMPLEMENTATIONS.md

As far as I understand the real challenge is not the OpenAPI generation part (some of the libraries on the above linked page should be able to do that), but mapping proto (particularly all of its types, including well-known types) to OpenAPI.

I have high hopes for gnostic, but the governance model for that project is a bit vague (eg. it's an important dependency for Kubernetes, so some changes are held back until they get prepared; it looks like it's primary role is to serve google API purposes).

A community governed project would certainly have some advantages.

Anyway, fingers crossed that gnostic "just works". 😄

batistein commented 1 year ago

any news on this topic? @sagikazarmark did you solved the problem?

DGuhr commented 1 year ago

I also hope this gets implemented anytime in the future. Sounds really like a big task, though.

Based on @ianrose14 comment I worked around it with this semi-automatic make target for now. It uses the docker image of swaggerapi/swagger-codegen-cli-v3 to generate openapi v3 files from the swagger.json that's generated by the openapiv2 plugin. Just leaving it here in case anyone needs the same.

Razikus commented 1 year ago

like @DGuhr mentioned

To convert you can use this: (execute where YOURTARGET.json is)

docker run --rm --name swagger_codegen -v $PWD:/opt/mnt:Z -w /opt/mnt swaggerapi/swagger-codegen-cli-v3:3.0.41 generate -i ./YOURTARGET.json -l openapi -o .
johanbrandhorst commented 1 year ago

This would be pretty cool to add to our docs, would you be willing to create a PR for this? Maybe a new section in https://github.com/grpc-ecosystem/grpc-gateway/blob/main/docs/docs/mapping/customizing_openapi_output.md?

iangudger commented 1 year ago

As of openapi-generator v6.6.0, the Python generator no longer supports OpenAPI 2.

Your spec version of 2.0 is too low. python only works with specs with version >= 3.X.X. Please use a tool like Swagger Editor or Swagger Converter to convert your spec to v3

What would be required to update to OpenAPI 3?