Parsing error while using nats

Hello everyone,

My team and I are quite new to krakend and we are having trouble connecting our back-end to the gateway. We are using nats for communicaiton. This is our gateway config:

{
  "version": 2,
  "timeout": "1s",
  "name": "account-service",
  "endpoints": [
    {
      "endpoint": "/test",
      "method": "POST",
      "headers_to_pass": ["Authorization", "Content-Type"],
      "backend": [
        {
          "host": ["nats://"],
          "disable_host_sanitize": true,
          "extra_config": {
            "github.com/devopsfaith/krakend-pubsub/publisher": {
              "topic_url": "create-account"
            }
          }
        }
      ]
    }
  ]
}

The error we are getting is this

kraken_1           | [GIN] 2021/05/30 - 15:29:53 | 200 |       172.1µs |      172.19.0.1 | POST     "/test"
account-service_1  | SyntaxError: Unexpected token  in JSON at position 0
account-service_1  |     at JSON.parse (<anonymous>)
account-service_1  |     at Client.processMsg (/app/node_modules/.pnpm/nats@1.4.12/node_modules/nats/lib/nats.js:1460:24)
account-service_1  |     at Client.processInbound (/app/node_modules/.pnpm/nats@1.4.12/node_modules/nats/lib/nats.js:1340:14)
account-service_1  |     at Socket.<anonymous> (/app/node_modules/.pnpm/nats@1.4.12/node_modules/nats/lib/nats.js:852:10)
account-service_1  |     at Socket.emit (events.js:376:20)
account-service_1  |     at addChunk (internal/streams/readable.js:309:12)
account-service_1  |     at readableAddChunk (internal/streams/readable.js:284:9)
account-service_1  |     at Socket.Readable.push (internal/streams/readable.js:223:10)
account-service_1  |     at TCP.onStreamRead (internal/stream_base_commons.js:188:23)
account-service_1  | [Nest] 105   - 05/30/2021, 3:29:53 PM   [RpcExceptionsHandler] The first argument must be of type string or an instance of Buffer, ArrayBuffer, or Array or an Array-like Object. Received undefined +502ms
account-service_1  | TypeError [ERR_INVALID_ARG_TYPE]: The first argument must be of type string or an instance of Buffer, ArrayBuffer, or Array or an Array-like Object. Received undefined
account-service_1  |     at Function.from (buffer.js:333:9)
account-service_1  |     at Object.hash (/app/node_modules/.pnpm/argon2@0.27.2/node_modules/argon2/argon2.js:38:42)
account-service_1  |     at async Function.createHash (/app/dist/accounts/models/password.model.js:26:22)
account-service_1  |     at async CreateAccountHandler.execute (/app/dist/accounts/commands/handlers/create-account.handler.js:47:26)
account-service_1  |     at async AccountsController.createAccount (/app/dist/accounts/accounts.controller.js:41:23)
account-service_1  |     at async /app/node_modules/.pnpm/@nestjs+microservices@7.6.17_81ebe84b585eae5955cca773bce1c4c7/node_modules/@nestjs/microservices/context/rpc-proxy.js:11:32
account-service_1  |     at async ServerNats.handleEvent (/app/node_modules/.pnpm/@nestjs+microservices@7.6.17_81ebe84b585eae5955cca773bce1c4c7/node_modules/@nestjs/microservices/server/server.js:63:32)

We are not sure why we are getting this error. The data we send to the gateway is in the body of the post request, the content-type is application/json. We don't know how this http request gets transformed to a nats message within kraken. Is there a way to tell kraken to take the body of the request and use it in the nats message?

I don't know what is your backend expecting, but you can check what is krakend sending to nats by adding another endpoint subscribed to the same topic:

{
  "version": 2,
  "timeout": "1s",
  "name": "account-service",
  "endpoints": [
    {
      "endpoint": "/test",
      "method": "POST",
      "headers_to_pass": ["Authorization", "Content-Type"],
      "backend": [
        {
          "host": ["nats://"],
          "disable_host_sanitize": true,
          "extra_config": {
            "github.com/devopsfaith/krakend-pubsub/publisher": {
              "topic_url": "create-account"
            }
          }
        }
      ]
    },
    {
      "endpoint": "/test",
      "backend": [
        {
          "host": ["nats://"],
          "disable_host_sanitize": true,
          "extra_config": {
            "github.com/devopsfaith/krakend-pubsub/subscriber": {
              "subscription_url": "create-account"
            }
          }
        }
      ]
    }
  ]
}

$ curl -id'{"message":1}' localhost:8080/test
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
X-Krakend: Version 1.3.0
X-Krakend-Completed: false
Date: Thu, 03 Jun 2021 14:40:33 GMT
Content-Length: 4

null%
$ curl -i localhost:8080/test
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
X-Krakend: Version 1.3.0
X-Krakend-Completed: true
Date: Thu, 03 Jun 2021 14:40:39 GMT
Content-Length: 13

{"message":1}%

Hello again, we did try your config file, everything's working fine. The issue arises when we try to read the incoming message via nats js client. Here's our client code

import { connect, StringCodec } from "nats";

const nc = await connect({ servers: "nats://nats:4222" });
console.log(nc.subscribe);
const sc = 

const sub = nc.subscribe("create-account");

for await (const m of sub) {
  console.log(`[${sub.getProcessed()}]: ${sc.decode(m.data)}`);
}

await nc.drain();

The output is this

 [1]: ����
 ����X-Forwarded-For
172.19.0.1X-Forwarded-Hostlocalhost:8080
User-AgentKrakenD Version 1.3.0
Content-Type!application/x-www-form-urlencoded
{"message":1}

Which throws a parsing error when we replace sc.decode with jsonc.decode (which is just a json decoder)

/app/node_modules/nats/lib/nats-base-client/error.js:98
        return new NatsError(m, code, chainedError);
               ^
NatsError: Bad JSON
     at Function.errorForCode (/app/node_modules/nats/lib/nats-base-client/error.js:98:16)
     at Object.decode (/app/node_modules/nats/lib/nats-base-client/codec.js:50:41)
     at file:///app/index.mjs:13:49
     at processTicksAndRejections (node:internal/process/task_queues:96:5) {
   code: 'BAD_JSON',
   chainedError: SyntaxError: Unexpected token  in JSON at position 0
       at JSON.parse (<anonymous>)
       at Object.decode (/app/node_modules/nats/lib/nats-base-client/codec.js:47:29)
       at file:///app/index.mjs:13:49

When we run curl -i localhost:8080/test the output is exactly the same as yours

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
X-Krakend: Version 1.3.0
X-Krakend-Completed: false
Date: Fri, 04 Jun 2021 02:09:33 GMT
Content-Length: 4

null%

So I think the issue is caused because we are getting the whole http request in the nats message instead of the body. Is there a way to configure krakend to only forward the body of the incoming request?

@kaanyalti Hi, have you found an explanation for this issue in the meantime?

I'm having the same problem in our setup too: When I read the message from NATS (e.g. via nats-cli), the message contains these strange characters and some headers, followed by the body:

����

    ����X-Forwarded-Hostlocalhost:8082
User-AgentKrakenD Version 2.6.3
                               Content-Typeapplication/jsonX-Forwarded-For
                                                                          192.168.65.1��
��{"query":"query { ...

I can't really make sense of this structure and the control (?) characters...

krakend / krakend-pubsub

Parsing error while using nats #11