Getting 500 response for valid url

[benoitc]

can you provide more informations: logs, trace or anything that could help to reproduce the issue?

[idod]

Sorry, I got an error on hackney and not on curl. No I get it also on curl :) No issue.

[idod]

So there is an issue: when trying to get the url of "https://assets.superbalistcdn.co.za/500x720/filters:quality(75):format(jpg)/spree/product/s/p/spree2018september12picslgzh0650.jpg" with hacnkey I get an error, although it is working fine on curl / browser:

when getting with hackney: hackney:get(<<"https://assets.superbalistcdn.co.za/500x720/filters:quality(75):format(jpg)/spree/product/s/p/spree2018september12picslgzh0650.jpg">>). {ok,500, [{<<"Date">>,<<"Tue, 08 Jan 2019 14:11:53 GMT">>}, {<<"Content-Type">>,<<"text/html; charset=UTF-8">>}, {<<"Transfer-Encoding">>,<<"chunked">>}, {<<"Connection">>,<<"keep-alive">>}, {<<"Set-Cookie">>, <<"__cfduid=d8081ddfb7dd7ce93d45546477584bcc11546956713; expires=Wed, 08-Jan-20"...>>}, {<<"X-Varnish">>,<<"15930532">>}, {<<"Via">>,<<"1.1 varnish-v4, 1.1 google">>}, {<<"Alt-Svc">>,<<"clear">>}, {<<"CF-Cache-Status">>,<<"MISS">>}, {<<"X-Content-Type-Options">>,<<"nosniff">>}, {<<"Expect-CT">>, <<"max-age=604800, report-uri=\"https://report-uri.cloud"...>>}, {<<"Server">>,<<"cloudflare">>}, {<<"CF-RAY">>,<<"495f41806f2029dd-SEA">>}],

Ref<0.2804061591.1561329665.145241>}

When curl: curl -v "https://assets.superbalistcdn.co.za/500x720/filters:quality(75):format(jpg)/spree/product/s/p/spree2018september12picslgzh0650.jpg" -O /dev/null % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Trying 104.20.2.217...

• TCP_NODELAY set
• Connected to assets.superbalistcdn.co.za (104.20.2.217) port 443 (#0)
• ALPN, offering h2
• ALPN, offering http/1.1
• Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
• successfully set certificate verify locations:
• CAfile: /etc/ssl/cert.pem CApath: none
• TLSv1.2 (OUT), TLS handshake, Client hello (1): } [233 bytes data]
• TLSv1.2 (IN), TLS handshake, Server hello (2): { [96 bytes data]
• TLSv1.2 (IN), TLS handshake, Certificate (11): { [3229 bytes data]
• TLSv1.2 (IN), TLS handshake, Server key exchange (12): { [115 bytes data]
• TLSv1.2 (IN), TLS handshake, Server finished (14): { [4 bytes data]
• TLSv1.2 (OUT), TLS handshake, Client key exchange (16): } [37 bytes data]
• TLSv1.2 (OUT), TLS change cipher, Client hello (1): } [1 bytes data]
• TLSv1.2 (OUT), TLS handshake, Finished (20): } [16 bytes data]
• TLSv1.2 (IN), TLS change cipher, Client hello (1): { [1 bytes data]
• TLSv1.2 (IN), TLS handshake, Finished (20): { [16 bytes data]
• SSL connection using TLSv1.2 / ECDHE-ECDSA-CHACHA20-POLY1305
• ALPN, server accepted to use h2
• Server certificate:
• subject: OU=Domain Control Validated; OU=PositiveSSL Multi-Domain; CN=ssl876453.cloudflaressl.com
• start date: May 18 00:00:00 2018 GMT
• expire date: Mar 9 23:59:59 2019 GMT
• subjectAltName: host "assets.superbalistcdn.co.za" matched cert's "*.superbalistcdn.co.za"
• issuer: C=GB; ST=Greater Manchester; L=Salford; O=COMODO CA Limited; CN=COMODO ECC Domain Validation Secure Server CA 2
• SSL certificate verify ok.
• Using HTTP2, server supports multi-use
• Connection state changed (HTTP/2 confirmed)
• Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
• Using Stream ID: 1 (easy handle 0x7fa71f006000)

  GET /500x720/filters:quality(75):format(jpg)/spree/product/s/p/spree2018september12picslgzh0650.jpg HTTP/2 Host: assets.superbalistcdn.co.za User-Agent: curl/7.54.0 Accept: /

• Connection state changed (MAX_CONCURRENT_STREAMS updated)! < HTTP/2 200 < date: Tue, 08 Jan 2019 14:12:52 GMT < content-type: image/jpeg < content-length: 29939 < set-cookie: __cfduid=d0c7d00daab667fc4417daa3ad7e908a01546956772; expires=Wed, 08-Jan-20 14:12:52 GMT; path=/; domain=.superbalistcdn.co.za; HttpOnly < alt-svc: clear < cache-control: public, max-age=7776000 < cf-bgj: imgq:100 < cf-polished: origSize=33101 < etag: "022e2364f299e640a24e5995a280ad35b9e6591b" < expires: Mon, 08 Apr 2019 14:12:52 GMT < via: 1.1 varnish-v4, 1.1 google < x-varnish: 9712551 < cf-cache-status: HIT < accept-ranges: bytes < x-content-type-options: nosniff < expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct" < server: cloudflare < cf-ray: 495f42f4ae57ad6f-TLV < { [898 bytes data] 100 29939 100 29939 0 0 308k 0 --:--:-- --:--:-- --:--:-- 311k
• Connection #0 to host assets.superbalistcdn.co.za left intact
• malformed
• Closing connection -1

[aditrades]

Happens to me too :-\

[joeyfeldberg]

If you remove the "filters:quality(75):format(jpg)" part of the URL it works, so might be a parser issue?

[benoitc]

@joeyfeldberg good catch

Hackney will normalize the URL and encode ( and ):

5> hackney_url:normalize(Url).
{hackney_url,hackney_ssl,https,
             <<"assets.superbalistcdn.co.za">>,
             <<"/500x720/filters:quality(75):format(jpg)/spree/product/s/p/spree2018september12picslgzh0650.jpg">>,
             <<"/500x720/filters:quality%2875%29:format%28jpg%29/spree/product/s/p/spree2018september12picsl"...>>,
             <<>>,<<>>,"assets.superbalistcdn.co.za",443,<<>>,<<>>}

Normally the server must decode it. Anyway as a temporary solution @idod if you don't want to en,code the Url you can pass your own function:

9> hackney:request(get, Url, [], [], [{path_encode_fun, fun(U) -> U end}]).
{ok,200,
    [{<<"Date">>,<<"Wed, 09 Jan 2019 09:13:52 GMT">>},
     {<<"Content-Type">>,<<"image/jpeg">>},
     {<<"Content-Length">>,<<"33101">>},
     {<<"Connection">>,<<"keep-alive">>}, 
     {<<"Set-Cookie">>,
      <<"__cfduid=dc4abf8a53694e4c9c40839736259fa0c1547025231; expires=Thu, 09-Jan-20"...>>},
     {<<"Expires">>,<<"Tue, 09 Apr 2019 09:13:52 GMT">>},
     {<<"Etag">>,
      <<"\"022e2364f299e640a24e5995a280ad35b9e6591b\"">>},
     {<<"Cache-Control">>,<<"public, max-age=7776000">>},
     {<<"X-Varnish">>,<<"16710304">>},
     {<<"Via">>,<<"1.1 varnish-v4, 1.1 google">>},
     {<<"Alt-Svc">>,<<"clear">>},
     {<<"CF-Cache-Status">>,<<"MISS">>},
     {<<"Accept-Ranges">>,<<"bytes">>},
     {<<"X-Content-Type-Options">>,<<"nosniff">>},
     {<<"Expect-CT">>,
      <<"max-age=604800, report-uri=\"https://"...>>},
     {<<"Server">>,<<"cloudflare">>},
     {<<"CF-RAY">>,<<"4965ca53c879a899-CDG">>}],
    #Ref<0.2882158659.3792961537.99124>}

I will figure about a better way to handle it.

[benoitc]

fixed in master via 4a501a0eb6adcce3c6ea3d90287fa8da18f5cd09