luraproject / lura

Ultra performant API Gateway with middlewares. A project hosted at The Linux Foundation
https://luraproject.org
Other
6.35k stars 564 forks source link

AMQP Routing key not passed #339

Closed nico-swan-cygnus-labs closed 4 years ago

nico-swan-cygnus-labs commented 4 years ago

Describe the bug I have been trying every which way to include the routing key but when the message arrives at my exchange the routing key is empty. my exchange is CYGNUS-LABS-PROD my routing key is REQUEST.INBOUND.SITE my binding is REQUEST.INBOUND.#

env RABBIT_SERVER_URL=amqp://fmctktcu:***@fly.rmq.cloudamqp.com/fmctktcu

If I change the binding in RabbitMQ to just '#' Then it

To Reproduce Steps to reproduce the behaviour:

  1. Configuration used "backend": [ { "host": [ "rabbit://" ], "disable_host_sanitize": true, "extra_config": { "github.com/devopsfaith/krakend-pubsub/publisher": { "topic_url": "CYGNUS-LABS-PROD" }, "github.com/devopsfaith/krakend-amqp/produce": { "name": " "exchange": "CYGNUS-LABS-PROD", "routing_key": "REQUEST.INBOUND.SITE", "durable": true, "delete": false, "exclusive": false, "no_wait": true, "mandatory": true, "immediate": false } } } ]

Expected behaviour I'm expecting the when the message passes through the exchanges on rabbitMQ that it has a routing key.

Logs Parsing configuration file: /etc/krakend/krakend.json 2020/03/25 14:05:49 ERROR: unable to create the gologging logger: getting the extra config for the krakend-gologging module 2020/03/25 14:05:49 ERROR: unable to create the GELF writer: getting the extra config for the krakend-gelf module 2020/03/25 14:05:49 INFO: Listening on port: 8080 2020/03/25 14:05:49 DEBUG: creating a new influxdb client 2020/03/25 14:05:49 DEBUG: no config for the influxdb client. Aborting 2020/03/25 14:05:49 WARNING: influxdb: unable to load custom config 2020/03/25 14:05:49 WARNING: opencensus: no extra config defined for the opencensus module 2020/03/25 14:05:49 WARNING: building the etcd client: unable to create the etcd client: no config 2020/03/25 14:05:49 DEBUG: no config for the bloomfilter 2020/03/25 14:05:49 WARNING: bloomFilter: no config for the bloomfilter 2020/03/25 14:05:49 WARNING: no config present for the httpsecure module 2020/03/25 14:05:49 DEBUG: lua: no extra config 2020/03/25 14:05:49 DEBUG: AMQP: rabbit://: no amqp consumer defined 2020/03/25 14:05:49 ERROR: AMQP: getting the channel for rabbit:///provision_inbound_requests: AMQP scheme must be either 'amqp://' or 'amqps://' 2020/03/25 14:05:49 DEBUG: pubsub: subscriber (rabbit://): github.com/devopsfaith/krakend-pubsub/subscriber not found in the extra config 2020/03/25 14:05:49 INFO: registering usage stats for cluster ID 'tdpRzEpggWE3eyS5AY2Whmaf7WyN47NoEjMrNQU4d9w=' 2020/03/25 14:05:50 DEBUG: CEL: no extra config detected for backend / 2020/03/25 14:05:50 DEBUG: lua: no extra config 2020/03/25 14:05:50 DEBUG: CEL: no extra config detected for pipe /publish/ 2020/03/25 14:05:50 DEBUG: lua: no extra config 2020/03/25 14:05:50 INFO: JOSE: singer disabled for the endpoint /publish/ 2020/03/25 14:05:50 DEBUG: lua: no extra config 2020/03/25 14:05:50 INFO: JOSE: validator disabled for the endpoint /publish/ 2020/03/25 14:05:50 DEBUG: botdetector: no config defined for the module 2020/03/25 14:05:50 DEBUG: http-server-handler: no extra config [GIN] 2020/03/25 - 14:06:06 | 500 | 767.13098ms | 196.50.230.54 | POST /publish/ Error #01: pubsub (code=Unknown): rabbitpubsub: message returned from CYGNUS-LABS-PROD: NO_ROUTE (code 312)

kpacha commented 4 years ago

the routing_key is not intended to be hardcoded. it's referring to the name of a request parameter

    {
        "endpoint": "/produce/{route}",
        "method": "POST",
        "backend": [
        {
          "url_pattern": "/",
          "host": ["amqp://guest:guest@192.168.99.100:5672"],
          "disable_host_sanitize": true,
          "extra_config": {
            "github.com/devopsfaith/krakend-amqp/produce": {
              "name": "pending_users",
              "exchange": "events",
              "durable": true,
              "routing_key": "Route"
            }
          }
        }
        ]
    }
nico-swan-cygnus-labs commented 4 years ago

Thank you for the response this worked but there are some findings that i would like to share :)

  1. In the above config, the "routing_key" seems case sensitive. It must have a value of "Route" and not "route". If I use "routing_key": "route" instead of "routing_key": "Route" then it fails to add the routing key.

  2. When using the "github.com/devopsfaith/krakend-pubsub/publisher" and the host is set to "rabbit://" to utilize the use of the env var RABBIT_SERVER_URL. It still doesn't send the routing key. I have also tried to change the host to "amqp://" both with or without setting "github.com/devopsfaith/krakend-pubsub/publisher" (See configuration extract below)

  3. I have a question around security. Ideally it is not ideal saving the username and password in the amqp host string i.e. "host": ["amqp://guest:guest@192.168.99.100:5672"]. Is there a way to incorporate a product like Hashicorp Vault to get the credentials when a request is made?

"endpoints": [ { "endpoint": "/working/{route}", "method": "POST", "backend": [ { "url_pattern": "/", "host": ["amqp://guest:guest@192.168.99.100:5672"], "disable_host_sanitize": true, "extra_config": { "github.com/devopsfaith/krakend-amqp/produce": { "name": "provision_inbound_requests", "exchange": "CYGNUS-LABS-PROD", "durable": true, "routing_key": "Route" } } } ] }, { "endpoint": "/notworking/{route}", "method": "POST", "backend": [ { "url_pattern": "/", "host": ["amqp://guest:guest@192.168.99.100:5672"], "disable_host_sanitize": true, "extra_config": { "github.com/devopsfaith/krakend-amqp/produce": { "name": "provision_inbound_requests", "exchange": "CYGNUS-LABS-PROD", "durable": true, "routing_key": "route" } } } ] }, { "endpoint": "/notworking1/{route}", "method": "POST", "backend": [ { "url_pattern": "/", "method": "POST", "host": [ "rabbit://" ], "disable_host_sanitize": true, "encoding": "json", "extra_config": { "github.com/devopsfaith/krakend-pubsub/publisher": { "topic_url": "CYGNUS-LABS-PROD" }, "github.com/devopsfaith/krakend-amqp/produce": { "exchange": "CYGNUS-LABS-PROD", "name": "provision_inbound_requests", "durable": true, "routing_key": "Route" } } } ] } ]

kpacha commented 4 years ago
  1. you are right
  2. the pubsub package and the amqp are 2 totally different animals with different goals and features
  3. you could use a custom script to get your secrets from wherever you want and inject them into a settings file for the flexible configuration (https://www.krakend.io/docs/configuration/flexible-config/)
github-actions[bot] commented 2 years ago

This issue was marked as resolved a long time ago and now has been automatically locked as there has not been any recent activity after it. You can still open a new issue and reference this link.