search

chansen / p5-http-tiny

Tiny HTTP Client
https://metacpan.org/dist/HTTP-Tiny
53 stars 52 forks source link

->mirror not working with redirects #116

Closed masterbee closed 6 years ago

masterbee commented 6 years ago

@chansen does the ->mirror(...) allow for redirects? I am trying to download the following file from github.

https://github.com/jgm/pandoc/releases/download/2.3.1/pandoc-2.3.1-macOS.zip and HTTP::Tiny is croaking with an error message in connecting to the SSL. After some examinations of the headers, I noticed the above URL was actually a 302 (redirect) to the actual octet-stream provided but github.

I was surprised to see that ->mirror would fail on this.

Any ideas?

xdg commented 6 years ago

What error message did you receive?

It worked for me:

$ perl -MHTTP::Tiny -wE 'my $ht=HTTP::Tiny->new(); my $resp = $ht->mirror("https://github.com/jgm/pandoc/releases/download/2.3.1/pandoc-2.3.1-macOS.zip", "pandoc-2.3.1-macOS.zip"); use DDP; p($resp)'
\ {
    content     "",
    headers     {
        accept-ranges         "bytes",
        content-disposition   "attachment; filename=pandoc-2.3.1-macOS.zip",
        content-length        17377360,
        content-type          "application/octet-stream",
        date                  "Tue, 02 Oct 2018 21:56:13 GMT",
        etag                  ""7ed5993a8061e5895a1b665724d91b8a"",
        last-modified         "Sat, 29 Sep 2018 02:56:44 GMT",
        server                "AmazonS3",
        x-amz-id-2            "O+VqRJxiRpgJznOsdgOJhAXmGhdMYIMgjbhE5bbywnXiP+6SpQtXQRtLq8Jro19MJ8KxtADKwcI=",
        x-amz-request-id      "506D4692B7FB04C9"
    },
    protocol    "HTTP/1.1",
    reason      "OK",
    redirects   [
        [0] {
            content    "<html><body>You are being <a href="https://github-production-release-asset-2e65be.s3.amazonaws.com/571770/668f3600-c358-11e8-8f5c-fc51579c14e4?X-Amz-Algorithm=AWS4-HMAC-SHA256&amp;X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20181002%2Fus-east-1%2Fs3%2Faws4_request&amp;X-Amz-Date=20181002T215612Z&amp;X-Amz-Expires=300&amp;X-Amz-Signature=63d110d65f5aa79254a3281ff109797f5c057297929766c99e5e173b292ecb09&amp;X-Amz-SignedHeaders=host&amp;actor_id=0&amp;response-content-disposition=attachment%3B%20filename%3Dpandoc-2.3.1-macOS.zip&amp;response-content-type=application%2Foctet-stream">redirected</a>.</body></html>",
            headers    {
                cache-control               "no-cache",
                content-security-policy     "default-src 'none'; base-uri 'self'; block-all-mixed-content; connect-src 'self' uploads.github.com status.github.com collector.githubapp.com api.github.com www.google-analytics.com github-cloud.s3.amazonaws.com github-production-repository-file-5c1aeb.s3.amazonaws.com github-production-upload-manifest-file-7fdce7.s3.amazonaws.com github-production-user-asset-6210df.s3.amazonaws.com wss://live.github.com; font-src assets-cdn.github.com; form-action 'self' github.com gist.github.com; frame-ancestors 'none'; frame-src render.githubusercontent.com; img-src 'self' data: assets-cdn.github.com identicons.github.com collector.githubapp.com github-cloud.s3.amazonaws.com *.githubusercontent.com; manifest-src 'self'; media-src 'none'; script-src assets-cdn.github.com; style-src 'unsafe-inline' assets-cdn.github.com",
                content-type                "text/html; charset=utf-8",
                date                        "Tue, 02 Oct 2018 21:56:12 GMT",
                expect-ct                   "max-age=2592000, report-uri="https://api.github.com/_private/browser/errors"",
                location                    "https://github-production-release-asset-2e65be.s3.amazonaws.com/571770/668f3600-c358-11e8-8f5c-fc51579c14e4?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20181002%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20181002T215612Z&X-Amz-Expires=300&X-Amz-Signature=63d110d65f5aa79254a3281ff109797f5c057297929766c99e5e173b292ecb09&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Dpandoc-2.3.1-macOS.zip&response-content-type=application%2Foctet-stream",
                server                      "GitHub.com",
                set-cookie                  [
                    [0] "has_recent_activity=1; path=/; expires=Tue, 02 Oct 2018 22:56:12 -0000",
                    [1] "_octo=GH1.1.2045724746.1538517372; domain=.github.com; path=/; expires=Fri, 02 Oct 2020 21:56:12 -0000",
                    [2] "logged_in=no; domain=.github.com; path=/; expires=Sat, 02 Oct 2038 21:56:12 -0000; secure; HttpOnly",
                    [3] "_gh_sess=d3ZKbGw5TmsxWWNQU2JDQVVkOFUwZzIySERBVkthYWRqNG5kT09xYlFPRWcwYzZac0cwTVY5RXE4TDdwZzZrOVcvZVNjYTRxZTRWNnd0Um9aKzNJWENpbXdPZm5FQkMwLzNSai84S0RVNkU4YXp3MHFibzFBcG1DVkwvUHl0S3hsQ0NpelJ0RW9PQzBuUHVTSUVOdVF6QVg2dGVJQkxyOHl2ZkhqSGJGSlQwT0VVTE92akx4NEZmLzRSemxSWHcvLS1lclhoVEJYS2kxYzJDekNRcU9RYVhBPT0%3D--11df353259e23c8cb696a5d6cea15079dc6ad354; path=/; secure; HttpOnly"
                ],
                status                      "302 Found",
                strict-transport-security   "max-age=31536000; includeSubdomains; preload",
                transfer-encoding           "chunked",
                vary                        "X-PJAX",
                x-content-type-options      "nosniff",
                x-frame-options             "deny",
                x-github-request-id         "ECD4:3483:2DEBA37:4CFEAE7:5BB3E97C",
                x-request-id                "a031615a-6658-4f4c-9e93-50941ff60239",
                x-runtime                   0.064017,
                x-runtime-rack              0.074586,
                x-xss-protection            "1; mode=block"
            },
            protocol   "HTTP/1.1",
            reason     "Found",
            status     302,
            success    "",
            url        "https://github.com/jgm/pandoc/releases/download/2.3.1/pandoc-2.3.1-macOS.zip"
        }
    ],
    status      200,
    success     1,
    url         "https://github-production-release-asset-2e65be.s3.amazonaws.com/571770/668f3600-c358-11e8-8f5c-fc51579c14e4?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20181002%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20181002T215612Z&X-Amz-Expires=300&X-Amz-Signature=63d110d65f5aa79254a3281ff109797f5c057297929766c99e5e173b292ecb09&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Dpandoc-2.3.1-macOS.zip&response-content-type=application%2Foctet-stream"
}

$ ls -l pandoc-2.3.1-macOS.zip
-rw-r--r--+ 1 david  staff    17M Sep 28 22:56 pandoc-2.3.1-macOS.zip
masterbee commented 6 years ago

@xdg - yet I found out that it was my openssl installation on my Mac. The error was complaining about SSL and redirects.

No need to fix, the issue was on my machine, appreciate the follow-up thou.