roback
commented
1 week ago An example with the URL that caused me to take a look at this issue in the first place (https://groovefm.stream.laut.fm/groovefm?t302=2024-11-06_11-42-26&uuid=64608ac6-2e1d-4300-ab7f-024022b06701).
The URL above returns an audio stream that never ends, but if you configure the client with max_response_body_size_bytes, it will abort the stream after the configured number of bytes:
$ bundle exec ruby -rlogger -rtwingly/http -e 'logger = Logger.new($stdout); logger.level = :INFO; puts Twingly::HTTP::Client.new(logger: logger, base_user_agent: "").tap { |client| client.max_response_body_size_bytes = 1000 * 1000 }.get("https://groovefm.stream.laut.fm/groovefm?t302=2024-11-06_11-42-26&uuid=64608ac6-2e1d-4300-ab7f-024022b06701")'
I, [2024-11-07T08:27:54.053982 #62937] INFO -- request: source=upstream-request method=GET url=https://groovefm.stream.laut.fm/groovefm?t302=2024-11-06_11-42-26&uuid=64608ac6-2e1d-4300-ab7f-024022b06701 request_id= release=unknown_heroku_release_version
/Users/me/repos/twingly-http/lib/twingly/http.rb:115:in `rescue in http_response_for': Response body too large, exceeced the configured max size of 1000000 bytes. (Twingly::HTTP::ResponseBodySizeLimitExceededError)
from /Users/me/repos/twingly-http/lib/twingly/http.rb:101:in `http_response_for'
from /Users/me/repos/twingly-http/lib/twingly/http.rb:62:in `get'
from -e:1:in `<main>'
/Users/me/repos/twingly-http/lib/faraday/response_body_size_limit/middleware.rb:23:in `block in call': Response body too large, exceeced the configured max size of 1000000 bytes. (Faraday::ResponseBodySizeLimit::LimitExceededError)
from /Users/me/.gem/ruby/3.2.2/gems/faraday-net_http-1.0.1/lib/faraday/adapter/net_http.rb:113:in `block in perform_request'
...
Added a Faraday middleware which streams the body, counting the number of bytes it receives. If the body exceeds the configured maximum size (using the
max_response_body_size_bytes:argument), it raises an error.close #21