Support \t in Content-Type header

rg2011 commented 5 months ago

IoT Agent JSON version the issue has been seen with

3.1.0

Bound or port used (API interaction)

Southbound (Devices data API)

NGSI version

NGSIv2

Are you running a container?

Yes, I am using a contaner (Docker, Kubernetes...)

Image type

normal

Expected behaviour you didn't see

This is a follow-up to issue https://github.com/telefonicaid/iotagent-json/issues/759. The XML service integration still fails because the client includes a "\t" character in the Content-Type header.

When sending a measure with content-type "application/soap+xml;\tcharset=utf-8" to /iot/json/attrs/<name of attrib>, the request should be accepted according to https://github.com/telefonicaid/iotagent-json/pull/771

Unexpected behaviour you saw

The IoT Agent replies inmediately with "400 Bad Content".

This is because the type-is module, used internally by express, fails to match this content-type to application/soap+xml:

$ node
Welcome to Node.js v20.10.0.
Type ".help" for more information.
> const typeis = require('type-is').is
undefined
> const ctype = "application/soap+xml; action=\"urn:notificarEvento\"; \tcharset=utf-8"
undefined
> typeis(ctype, ["application/soap+xml"])
false
> typeis(ctype, ctype)
false
>

This causes the following checks to fail:

https://github.com/telefonicaid/iotagent-json/blob/db402996729536d42e94ce70101bdfd896a793ea/lib/bindings/HTTPBinding.js#L75-L83

https://github.com/telefonicaid/iotagent-json/blob/db402996729536d42e94ce70101bdfd896a793ea/lib/bindings/HTTPBinding.js#L109-L115

And also an internal test in the bodyparser-xml dependency, which performs its own content-type check, matching to ['*/xml', '+xml'] by default:

Removing the \t makes type-is recognize the content-type properly:

> const ctype2 = "application/soap+xml; action=\"urn:notificarEvento\"; charset=utf-8"
undefined
> typeis(ctype2, ["application/soap+xml"])
'application/soap+xml'
>

I did open an issue upstream: https://github.com/jshttp/type-is/issues/52

But I don't expect a fix to reach express anytime soon, so maybe we would need a workaround.

Steps to reproduce the problem

export TAB=$'\t'

curl -H "Content-Type: application/soap+xml;${TAB}charset=utf-8" "http://<iota json ip>:7897/iot/json/attrs/payload?i=<deviceid>&k=<apikey>" -d '<?xml version="1.0"?><body><attrib>value</attrib></body>"

Configs

environment:
    - IOTA_APPEND_MODE=true
    - IOTA_MONGO_HOST=iot-mongo
    - IOTA_MONGO_DB=iotajson
    - IOTA_SINGLE_MODE=false
    - IOTA_CB_HOST=iot-orion
    - IOTA_CB_NGSI_VERSION=v2
    - IOTA_AUTH_ENABLED=false
    - IOTA_AUTH_TYPE=keystone
    - IOTA_AUTH_HEADER=X-Auth-Token
    - IOTA_AUTH_HOST=172.17.0.1
    - IOTA_AUTH_PORT=5001
    - IOTA_AUTH_USER=iotagent
    - IOTA_AUTH_PASSWORD=***OFUSCATED***
    - IOTA_IOTAM_HOST=iot-iotagent-manager
    - IOTA_IOTAM_PORT=8082
    - IOTA_IOTAM_PATH=/iot/protocols
    - IOTA_IOTAM_AGENTPATH=
    - IOTA_IOTAM_PROTOCOL=IoTA-JSON
    - IOTA_IOTAM_DESCRIPTION=JSON_IoT_Agent_Node
    - IOTA_PROVIDER_URL=http://172.17.0.1:4052
    - IOTA_NORTH_PORT=4052
    - IOTA_HTTP_PORT=7897
    - IOTA_MQTT_USERNAME=iota
    - IOTA_MQTT_PASSWORD=***OFUSCATED***
    - IOTA_MQTT_HOST=iot-mosquitto
    - IOTA_MQTT_PORT=1883
    - IOTA_MQTT_QOS=2
    - IOTA_AMQP_DISABLED=true
    - IOTA_LOG_LEVEL=INFO
    - IOTA_REGISTRY_TYPE=mongodb
    - IOTA_DEFAULT_RESOURCE=/iot/json

Log output

{"name":"UNSUPPORTED_TYPE","message":"The request content didn't have the expected type [application/json, text/plain, application/octet-stream, application/soap+xml ]"}

AlvaroVega commented 5 months ago

I've tested with

req.get('content-type').includes('application/soap+xml')

but doesn't work.

rg2011 commented 5 months ago

I've tested with
req.get('content-type').includes('application/soap+xml')
but doesn't work.

Hi Alvaro. I understand you were able to replicate the issue with a test, or something like the snippet:

export TAB=$'\t'

curl -H "Content-Type: application/soap+xml;${TAB}charset=utf-8" "http://<iota json ip>:7897/iot/json/attrs/payload?i=<deviceid>&k=<apikey>" -d '<?xml version="1.0"?><body><attrib>value</attrib></body>"

But changing req.is to req.get('content-type').includes(...) was not enough to fix it. Is that right?

I am working in a forked branch, and found some other changes are needed too. Let me open a PR so we can discuss the details more easily.

telefonicaid / iotagent-json