I don't know if we'll see this issue fixed, but after a ton of debugging and headaches, I've finally tracked down what I think is the cause of all of my redirect issues in the typhoeus release 1.4.0 (from version 1.3.1). That also forced us to move from ethon version 0.12.0 to 0.16.0.
After looking at the ethon code diff, I had come to the same conclusion that other users in the typhoeus had, that the changes here had some unintended side effects. Please see https://github.com/typhoeus/typhoeus/issues/705 for more details.
Filing this issue here since this appears to be the package in question that is having undesired behaviour, which was introduced with https://github.com/typhoeus/ethon/pull/221.
Apologies if this is just being mis-used in typhoeus and this package is not at fault, but that repo has not seen any maintenance in a while (and those tickets haven't drawn any attention) so wanted to file this here in the hopes we may either get a proper workaround (aside from just ignoring the on_headers call since now it's not entirely useful), or even better, an update to fix that.
The current workaround I'm using is not ideal because it leads to the download flow being executed just to do basic content-length header checks, but it's manageable. This is my current on_headers method in order to work around this bug and still do the content-length check for most scenarios (when not involving a redirect):
req.on_headers do |response|
if (response.code != 301 && response.code != 302)
HttpHelper.assert_status_code!(response.code)
raise SomePackage::DownloadFailure, 'Invalid Download File Type' if response.headers['Content-Length'].to_i.zero?
end
end
For extra context though, this is not the case for on_complete, which will only emit after the final destination or max redirects are reached, which is the behaviour we'd expect.
Hello!
I don't know if we'll see this issue fixed, but after a ton of debugging and headaches, I've finally tracked down what I think is the cause of all of my redirect issues in the
typhoeus
release1.4.0
(from version1.3.1
). That also forced us to move fromethon
version0.12.0
to0.16.0
.After looking at the
ethon
code diff, I had come to the same conclusion that other users in thetyphoeus
had, that the changes here had some unintended side effects. Please see https://github.com/typhoeus/typhoeus/issues/705 for more details.Filing this issue here since this appears to be the package in question that is having undesired behaviour, which was introduced with https://github.com/typhoeus/ethon/pull/221.
Apologies if this is just being mis-used in
typhoeus
and this package is not at fault, but that repo has not seen any maintenance in a while (and those tickets haven't drawn any attention) so wanted to file this here in the hopes we may either get a proper workaround (aside from just ignoring theon_headers
call since now it's not entirely useful), or even better, an update to fix that.The current workaround I'm using is not ideal because it leads to the download flow being executed just to do basic
content-length
header checks, but it's manageable. This is my currenton_headers
method in order to work around this bug and still do thecontent-length
check for most scenarios (when not involving a redirect):For extra context though, this is not the case for
on_complete
, which will only emit after the final destination or max redirects are reached, which is the behaviour we'd expect.