copyfrom `Uncaught exception: Error: incorrect header check`

[simon-contreras-deel]

If you request a gzipped copyfrom sending a bad formatted gzip body (at least, we think the cause is a bad formatted gzip body), the API crashes. In the log you can see:

Uncaught exception: Error: incorrect header check
    at Zlib._handle.onerror (zlib.js:370:17)

The error was reported by @rafatower who could add more info

[rafatower]

It can be reproduced with the CARTO Python SDK at this commit: https://github.com/CartoDB/carto-python/pull/87/commits/bd19b8154b49947b246bec7cba3c2612509f0f3f

by executing the examples:

cd carto-python/examples
virtualenv env
source env/bin/activate
pip install -r requirements.txt
[...]
python copy_example.py
[...]
send: 'POST /api/v2/sql/copyfrom?q=COPY+copy_example+%28the_geom%2C+name%2C+age%29+FROM+stdin+WITH+%28FORMAT+csv%2C+HEADER+true%29&api_key=730162d337594b567fa5e51919cac5e3061a7197 HTTP/1.1\r\nHost: development.localhost.lan:8080\r\nConnection: keep-alive\r\nAccept: */*\r\nUser-Agent: python-requests/2.19.1\r\nTransfer-Encoding: chunked\r\nContent-Type: application/octet-stream\r\nContent-Encoding: gzip\r\n\r\n'
send: '65'
send: '\r\n'
send: 'x\x01+\xc9H\x8dOO\xcd\xcf\xd5\xc9K\xccM\xd5ILO\xe5\n\x0e\xf2t\xb15162\xb3\x0e\xf0\xf7\xf4\x0b\xd1\xd0542S05\xd1\xd4\xf1\xcb/*\xc9P\x08O-.\xd1\xb1\xb0\xc4Tfi\xa6`\x0cT\x15\x9c_\nT\xe5\x9a\x08Te\x89E\x95\x99\x82\xae\x91)TUj\x11X]j\x91\x8e\xa1\x91\t\x00\x99V&\x90'
send: '\r\n'
send: '0\r\n\r\n'

[rafatower]

I started working on this, by adding a test for this case: https://github.com/CartoDB/CartoDB-SQL-API/pull/530

[rafatower]

and... fixed! :smile:

@oleurud I'm handing this over to RT, in case you can handle the release and deployment (not super urgent but let's keep an eye, cause bad clients - like mine before fixing the compression - can harm a little).

@CartoDB/rt-managers could you please follow-up on this? thanks!

[simon-contreras-deel]

I have found a detail to change. I do it? On the other hand, do you think it's good that we deploy it on Monday? Is it too late?

[rafatower]

Go for it! Monday works for me. Thanks a lot! :slightly_smiling_face:

[simon-contreras-deel]

:rocket: in production