Support for message level security

pleothaud commented 5 years ago

As expressed in issues #1464 and #1881 there is a need for describing signed and encrypted requests and response using OpenAPI. I would like to propose the following mechanisms to support message level security in OAS.

This proposal comes after some preliminary works summarized in the Google doc at https://docs.google.com/document/d/13THkz7867blJ464ohpz9MUrhMQp-28Cx77JvDHlyurc from OAI Google Drive (I can give access to this doc to who needs it)

The problem with the proposal in this document is that it was too complex and did not follow the Open API philosophy, so I tried here to put in place something more standard, mimicking the Security Scheme / Security Requirement mechanism people are used to.

Let's take an example that shows how you could describe required requests signature and provided responses signature using JWT tokens (as you can see it uses the key description mechanism proposed by @whitlockjc in issue #1881) :

# ...
components:

    keys:
        # Keys taken from Google's OIDC Discover Document (https://accounts.google.com/.well-known/openid-configuration)
        google-oauth-v3-1:
            description: "JSON Web Key used by Google for signing JWTs: https://www.googleapis.com/oauth2/v3/certs#/keys/0"
            type: JWK
            metadata:
                kid: 0905d6f9cd9b0f1f852e8b207e8f673abca4bf75
                e: AQAB
                kty: RSA
                alg: RS256
                n: yyeEmeK35F8P54ozfpsF79n59ZsOrcZdxQWsxrzm0qjdA5r_b-be-cQnWAw_2AoGdeWHX-Cz7uPFDMdEwzLGlpv3SELi34h8PkzjyO7xlbhsNs-ICnqUyUTA7CovKtpJ47PjiQnXcaRNCFUQbli8VlEqbVLuqFjC98igICpNYR-iiVIm0VCFtkq0p8vf1yQ493Pnx2Bm8fUx6SkeJ7wKPWQq_K4e6ZH40JWLk6c1U9W5qPKeckevdNLrdZY5lsTZ5zrRvuRBoIeZfp9bKSZGMtEja4xSCDKLrkcpb4qf6Ywx9rsZ4b8eHSLpVvUzNsj3GS7qK5flHzoccovhPVBbbQ
                use: sig

    messageSecurityOperations:
        signedJWTrequestOperation:
            securityType: signedJWTRequest
                alg:
                    - PS256
                    - PS512
                pubKeyRef:
                    - jku+kid
                mandatoryStandardClaims:
                    - iss
                    - aud
                    - iat
                    - exp
                    - jti

        signedJwtResponseOperation:
            securityType: signedJWTResponse
                pubKey: 
                    $ref: '#/components/keys/google-oauth-v3-1'

    messageSecuritySchemes:
        highSensitivityScheme-PostPutPatch:
            requestSecurity:
                $ref: '#/components/messageSecurityOperations/signedJWTrequestOperation'
            responseSecurity:
                200:
                    $ref: '#/components/messageSecurityOperations/signedJwtResponseOperation'
# ...

As you can see, there are two objects added in the components top level object:

messageSecurityOperations, which is a Map of messageSecurityOperations describing the elementary signature and encryption operations that are required on API requests or performed as API responses
messageSecuritySchemes which is a Map of messageSecuritySchemes, each security scheme stating which messageSecurityOperation must be applied to the request before consuming the API and which messageSecurityOperation will be applied to the operation's responses, based on the response HTTP status code (all of these referencing the messageSecurityOperations declared in the messageSecurityOperations object

Once these objects declared in the components object, it becomes possible to reference a messageSecurityScheme directly from an Operation, just like this:

# ...
paths:
    /pets:
        post:
            description: Creates a new pet in the store.  Duplicates are allowed
            operationId: addPet
            messageSecurity: highSensitiveProfile-PostPutPatch
            requestBody:
                description: Pet to add to the store
                required: true
                content:
                    application/jwt:
                        schema:
                            $ref: '#/components/schemas/NewPet'
            responses:
                '200':
                    description: pet response
                    content:
                        application/jwt:
                            schema:
                                $ref: '#/components/schemas/Pet'
                default:
                    description: unexpected error
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/Error'
# ...

where the messageSecurity: highSensitivityScheme-PostPutPatch field of the post operation object specifies which message security scheme to apply to the operation (on requests and responses).

There are a few message level security operations that could be described this way:

Request Security Schemes

JWSRequest (standard JWS compact signature)
JWSDetachedRequest (request body unchanged, signature passed in a HTTP header)
SignedJWTRequest (signed JWT)
HTTPSigRequest (HTTPSignature standard description)
EncryptedJWTRequest (encrypted JWT)
JWERequest (standard compact JWE encrypted message)

Response Security Schemes

JWSResponse
JWSDetachedResponse
SignedJWTResponse
HTTPSigResponse
EncryptedJWTResponse
JWEResponse

If you feel comfortable with the presented structure of message level security description I can create separate issues for each of these message level security operations, specifying mandatory and optional attributes that could describe these operations.

Happy to discuss!

cyberphone commented 5 years ago

It is correct that https://datatracker.ietf.org/doc/draft-cavage-http-signatures is the only of the described schemes supporting REST? UK's Open Banking signature scheme does not include URI and Method although they are fundamental parts of a REST request.

pleothaud commented 5 years ago

HI Anders,

Yes at the moment it is the only scheme supporting URI/Method/HTTP headers signatrure available (while still in draft).

There was a proposal from Richer, Bradley and Tschofenig from the Oauth WG at IETF about that (https://tools.ietf.org/html/draft-ietf-oauth-signed-http-request-03) but it's been expired for 3 years now... Seems that IETF guys did not like the needed canonicalization of the HTP request :-)

As you know I'm thinking of proposing to the OAuth WG something simpler, based on the fact that you add Method + URI + a selection of HTTP headers in the JOSE header and then use the detached unencoded payload mechanism. Just running after time to write this down, as always :-(

Thanks,

Philippe

cyberphone commented 5 years ago

Hi Philippe, You will anyway need to canonicalize the URI and the HTTP headers. Here is my take on the latter: https://tools.ietf.org/html/draft-rundgren-signed-http-requests-00#section-6.8

BTW, canonicalization of the JSON body is not dead, it is just resting a bit :) It works like a charm on 5 platforms so the team behind it is currently trying to establish an "informational" RFC. This is not a "full standard" but I think it suffices for getting it off ground.

pleothaud commented 5 years ago

Mmmm no actually and that's the beauty of it :-)

As the info from the HTTP message you want to integrity protect is written in the JOSE header to some extent the canonicalization process lays in how you store the information in the JOSE header.

So when you verify the signature of the JWS you ensure that all the info in the JOSE header was not modified during transit.

Then all you've got to do is verifying that this info semantically matches the received HTTP message!

This has to be refined, a proper definition of what "semantically matches" means should be provided, but this should work and would be so easier, IMHO

cyberphone commented 5 years ago

This has to be refined, a proper definition of what "semantically matches" means should be provided, but this should work and would be so easier, IMHO

This is the URI and Header canonicalization I referred to. The data is created by the sender but must be compared by the receiver using "live" HTTP data from the server. The algorithm I showed is essentially the same as HTTP Signatures uses. I believe this part was the reason the previous attempt you mentioned didn't got through. Some people claim this is impossible since proxies modify data. Since HTTP Signatures and a similar solution from Amazon actually is heavily used, this is obviously exaggerated. Or to be more correct: Requiring that everything must work on any platform without adjustments is counter productive. Android Apps typically does not run on all versions of Android either.

The difference between detached JSON and canonicalized JSON is another discussion. They both have downsides. Detached JSON is a loss in the proxy world, for WebSockets and doesn't lend itself to serialization.

pleothaud commented 5 years ago

Semantic match is a different beast than byte to byte equality after transformation required by canonicalization processes. Set-Cookie addition by reverse proxies is an example of problem with canonicalization (and BTW as Set-Cookie expires is an unquoted RFC 822 date containing a comma, concatenating Set-Cookie values with a comma can be a problem: this is why you generate a Set-Cookie header by cookie) that can be easily overcome by semantic matching.

Anyway, this discussion about HTTP Signature is not totally in the scope of this issue :-)

I'll send you a PM about this.

cyberphone commented 5 years ago

I think I understand what you are meaning now.

You sign HTTP header data as is but in a Base64Url-safe way
After verifying the signature you try to match the signed data with the actual potentially somewhat changed data

I would be a bit cautious about trying to standardize such a scheme. If the data is HTTP-compliant it should be possible to canonicalize the data the way HTTP Signatures, Amazon and I suggest. If it is not HTTP-compliant, I wouldn't bother. If an intermediary adds a Set-Cookie I don't see how that could work with any scheme unless you treat each cookie as a separate item. Maybe that is a better approach; consider all concatenated headers as separate elements during canonicalization?

albertvp commented 4 years ago

Hi all, is there any conclusion from the meeting?

I think that encryption keys management should be treated as a separate service, like user session. But instead of Authorization headers, where you can specify a global token, might be nice to select or send a keyID within the request. This assumes that the dynamic specification of the key can switch to a different security level. If no key provided then no encryption is done.

Another important thing for me is how to specify which fields are encrypted and which are "public". So every attribute that contains sensitive information has to be included on the encrypted part, and the rest will be raw to perform searches with no key. A really cool thing might be to include in the ui-docs the payload transformation process (encrypt/decrypt) by default, so you still have inputs where to put raw values and decrypt the response to show again raw values.

I'm not really familiar to JOSE, but hope to provide another point of view, thank you!

MikeRalphson commented 1 year ago

Adding an updated IETF draft link - https://datatracker.ietf.org/doc/draft-ietf-httpbis-message-signatures/

OAI / OpenAPI-Specification

Support for message level security #1953