search

openfaas / nats-queue-worker

Queue-worker for OpenFaaS with NATS Streaming
https://docs.openfaas.com/reference/async/
MIT License
128 stars 59 forks source link

Adds http signatures to callbacks #56

Closed ewilde closed 2 years ago

ewilde commented 5 years ago

Resolves #48

PRs that should be merged first:

Testing

With signing keys configured (new way) on docker swarm

  1. Deploy openfaas gateway from this openfaas/faas#1101
  2. Deploy this PRs version of nats-queue-worker
  3. Deploy this function: https://github.com/ewilde/of-cloud-examples/tree/master/verify-go
  4. Deploy this function: https://github.com/ewilde/of-cloud-examples/tree/master/verify-node
  5. Deploy this function: https://github.com/ewilde/of-cloud-examples/tree/master/verify-python
  6. Deploy this function: https://github.com/ewilde/of-cloud-examples/tree/master/echo
  7. Execute command async:

curl http://localhost:8080/async-function/echo -d 'hello' -H "X-Callback-Url: http://gateway:8080/function/verify-go"

  1. Tail the logs of nats-queue-worker

    Request for echo.
[#2] Received on [faas-request]: 'sequence:2 subject:"faas-request" data:"{\"Header\":{\"Accept\":[\"*/*\"],\"Content-Length\":[\"5\"],\"Content-Type\":[\"application/x-www-form-urlencoded\"],\"User-Agent\":[\"curl/7.54.0\"],\"X-Call-Id\":[\"1e4aff58-e3ad-4e70-8927-5697965104b1\"],\"X-Callback-Url\":[\"http://gateway:8080/function/verify-go\"],\"X-Start-Time\":[\"1552687306808763300\"]},\"Host\":\"localhost:8080\",\"Body\":\"aGVsbG8=\",\"Method\":\"POST\",\"Path\":\"\",\"QueryString\":\"\",\"Function\":\"echo\",\"CallbackUrl\":{\"Scheme\":\"http\",\"Opaque\":\"\",\"User\":null,\"Host\":\"gateway:8080\",\"Path\":\"/function/verify-go\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"}}" timestamp:1552687306809674900 redelivered:true '
Callback to: http://gateway:8080/function/verify-go
Wrote 6 Bytes
200 OK
Posted result: 200
Posting report - 202

    ☝️ notice the Posted result is 200

  2. Tail the logs of the verifier function

    
2019/03/15 22:27:58 stderr: 2019/03/15 22:27:58 Signature:
AnR29Y67NyCratkpLZ3NvNwb8hx+pUD8zE6glAQMzw/UXi8pF3kMmYoMTjTRqwt6SNHSi315dKoUsyksH/ACOG+rIOmBcOj0h8G17XECBoDogGSNCGeZk7w/acYMXng56MhlUxNEsSi7ep++JwnmZz7catdnO0+7R2OOpALmsKIgTrX9GXJTk4x44A/v5EQaBhPjq0YSZ5gmB41n/jLTkOW9ptu1m4q99y+8E02OOpBaaRca7fI4guAXFqaAuEhaecB10c3gI80yqN87RC3aQdcDT7MflzvAx9ZKHTkLRCYUIcOwr2+AltYFiFppihUkNuhO3jNdtXWabCjxI5DsQQ==
2019/03/15 22:27:58 Signing string:
(request-target): post /function/verify-go
host: gateway:8080
date: Fri, 15 Mar 2019 22:27:58 GMT
content-type: application/x-www-form-urlencoded
digest: SHA-256=WJG1tSLV3whtD/CxEPvZ0hu0/HFjrzTQgoai6Eb2vgM=
content-length: 6
2019/03/15 22:27:58 Public key:
-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAwzN2WML3CelVpYSUto2h
c6bpqEYcxyGRhtPfvgH/YuU5XXOHDwWjlf6cgdH2OUUXi9xQ8S1Tj5FlH26fXNk+
HzBmPEsYdFArINY1i8R7LGGdWdsniol/Cn6WQwllUzm4rG+f0bQRL7h0FcH+aL6E
pNhaR42lmKCmScURt7EFMQOijA8z2TQvJ9lw7isXbzPtbaQxOkK8RyGN95KDXByI
oQynmVuiGTTsUqiMKMxhQXQbCrfUK1SX75Dlx2vRecWZE9CV6ijZAwH+F/BrDf1z
Eodzc1NnAC0Ui0LJf3wJ0c/9bAtiMSUu/2Zl8law93ME3sSZQh/q2x0QFnxQ2nYt
GQIDAQAB
-----END PUBLIC KEY-----

2019/03/15 22:27:58 Verified OK

☝️ notice  Verified OK and all the signing verification details

### Without signing keys configures (backwards compatibility)
If you deploy the new version of nats queue worker, but have not configured keys in the environment, it should fallback to unsigned messages

1. Nats queue worker prints out warning message

Http signatures disabled. Warning callback messages will not be signed missing private key: /run/secrets/http-signing-private-key Loading basic authentication credentials Connect: nats://nats:4222 Subscribing to: faas-request at nats://nats:4222 Wait for 5m5s Listening on [faas-request], clientID=[faas-worker-94731c9f0ecc], qgroup=[faas] durable=[]


2. Messages are still delivered to callback

[#2] Received on [faas-request]: 'sequence:2 subject:"faas-request" data:"{\"Header\":{\"Accept\":[\"/\"],\"Content-Length\":[\"5\"],\"Content-Type\":[\"application/x-www-form-urlencoded\"],\"User-Agent\":[\"curl/7.54.0\"],\"X-Call-Id\":[\"5a7e70e2-8aa5-4da0-b44c-6b2f68218bff\"],\"X-Callback-Url\":[\"http://gateway:8080/function/verify-go\"],\"X-Start-Time\":[\"1552689418027066400\"]},\"Host\":\"localhost:8080\",\"Body\":\"aGVsbG8=\",\"Method\":\"POST\",\"Path\":\"\",\"QueryString\":\"\",\"Function\":\"echo\",\"CallbackUrl\":{\"Scheme\":\"http\",\"Opaque\":\"\",\"User\":null,\"Host\":\"gateway:8080\",\"Path\":\"/function/verify-go\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"}}" timestamp:1552689418027743800 ' Wrote 6 Bytes 200 OK Callback to: http://gateway:8080/function/verify-go Posted result: 200 Posting report - 202


### With signing keys configured (new way) on k8s

1. Clone and setup environment

!/usr/bin/env bash

cd tmp git clone git@github.com:ewilde/faas-netes.git cd faas-netes git checkout http-signatures

helm upgrade openfaas --install ./chart/openfaas --namespace openfaas --set functionNamespace=openfaas-fn --set http_signatures=true --set gateway.image=ewilde/gateway:http-signatures --set queueWorker.image=ewilde/http-signatures`

faas deploy -f https://raw.githubusercontent.com/ewilde/of-cloud-examples/k8s/verify-go.yml faas deploy -f https://raw.githubusercontent.com/ewilde/of-cloud-examples/k8s/echo.yml


2. Tail verify function logs
`kubetail  verify-go   -n openfaas-fn`

3. Send async message
`curl http://(minikube ip):31112/async-function/echo -d 'hello' -H "X-Callback-Url: http://gateway.openfaas:8080/function/verify-go"`

4. Look for success message: Verified OK

[verify-go-67854bfb-d9kzk] (request-target): post /function/verify-go [verify-go-67854bfb-d9kzk] host: gateway.openfaas:8080 [verify-go-67854bfb-d9kzk] date: Sun, 07 Apr 2019 21:25:22 GMT [verify-go-67854bfb-d9kzk] content-type: application/x-www-form-urlencoded [verify-go-67854bfb-d9kzk] digest: SHA-256=WJG1tSLV3whtD/CxEPvZ0hu0/HFjrzTQgoai6Eb2vgM= [verify-go-67854bfb-d9kzk] content-length: 6 [verify-go-67854bfb-d9kzk] 2019/04/07 21:25:22 Public key: [verify-go-67854bfb-d9kzk] -----BEGIN PUBLIC KEY----- [verify-go-67854bfb-d9kzk] MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAuLzV66O5hAT2NJzwFJML [verify-go-67854bfb-d9kzk] ssOSdlKtErBrsWqzv1GTcy++TJ4/fBD3QuWf54vJYrVFTV7HEFtOE/Hcear2gVRo [verify-go-67854bfb-d9kzk] qgqL3oDA06rYnauHsxYewWyO0G2Z65L4grjNzdpd3aFRGKI6oBbS8UZ6baVvxO9w [verify-go-67854bfb-d9kzk] KEzCOf8/do4co3PjQ5JHh27bNgn8nxQ5sQRK2t4xBJcRFXhzPSVy3mWsdbsjsp12 [verify-go-67854bfb-d9kzk] Oc1RgT96+dmAfu11l06JE2a84E+E0d0rxLyRZYwCeLNRdt3FhFj4/KMrux+kNcQa [verify-go-67854bfb-d9kzk] KdUfI8oM6nGSVDNd54psx2sQJcufBE4y+KBOfI1Sm/oGUl6y0ujNUIbwSN4AXM8K [verify-go-67854bfb-d9kzk] aQIDAQAB [verify-go-67854bfb-d9kzk] -----END PUBLIC KEY----- [verify-go-67854bfb-d9kzk] [verify-go-67854bfb-d9kzk] 2019/04/07 21:25:22 Verified OK



## TODO:
- [x] Sign callback messages
  - [x] Load private key from secrets
  - [x] Create signer method
  - [x] Test signer method
  - [x] Test we can verify a signed request
  - [x] Wire signer method to callback process
- [x] Manual testing

<!--- Provide a general summary of your changes in the Title above -->

## Description
Adds http signatures to callbacks see: #48 

## How Has This Been Tested?
<!--- Please describe in detail how you tested your changes. -->
<!--- Include details of your testing environment, and the tests you ran to -->
<!--- see how your change affects other areas of the code, etc. -->

## Types of changes
<!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Checklist:
<!--- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
- [x] My code follows the code style of this project.
- [x] My change requires a change to the documentation.
- [ ] I have updated the documentation accordingly.
- [x] I've read the [CONTRIBUTION](https://github.com/openfaas/faas/blob/master/CONTRIBUTING.md) guide
- [x] I have signed-off my commits with `git commit -s`
- [x] I have added tests to cover my changes.
- [ ] All new and existing tests passed.
alexellis commented 5 years ago

Hi Ed,

Thanks for the instructions

Does your example need to have a namespace configurable?

    resp, err := http.Get("http://gateway:8080/certificates/callback")

This address on Kubernetes will be: gateway.openfaas (was it tested on Kubernetes?)

I'd like a CLI command to run to deploy the functions too for easy testing i.e.

git clone https://
faas-cli deploy --filter fn-x

Alex

ewilde commented 5 years ago

@alexellis test instructions updated to include faas deploy etc... Please see section above With signing keys configured (new way) on k8s