lostisland / faraday

Simple, but flexible HTTP client library, with support for multiple backends.
https://lostisland.github.io/faraday
MIT License
5.73k stars 976 forks source link

same request produces different response bodies (body inconsistently returns instance of Hash or String) #1388

Open ksilex opened 2 years ago

ksilex commented 2 years ago

Basic Info

Issue description

seems like when requests done in parallel response body doesn't get parsed

Steps to reproduce

def run1
    req = Faraday.new(config[:url]) do |conn|
      conn.response :json, content_type: %r{application/json}
    end
    req.get('/') do |conn|
      conn.body = body
    end
end

Screenshot_2

also seen similar problem here: https://stackoverflow.com/questions/67182790/error-response-not-parsed-by-faraday-despite-adding-json-middleware except no middlewares were used

olleolleolle commented 2 years ago

👋 Can you attempt with a higher version of 1.x series?

olleolleolle commented 2 years ago

Update: Here's a Bundler inline example, which allows you to pick versions used and have the example actually runnable.

I guessed at some information, so I may be off: this program only returned [Hash, Hash...] for me.

(Edited the example.)

#!/usr/bin/env ruby

require 'bundler/inline'

gemfile do
  source 'https://rubygems.org'
  gem "faraday", "= 1.6.0"
  gem "faraday_middleware"
end

def run
  url = "https://httpbingo.org"
  body = "{}"
  req = Faraday.new(url) do |conn|
    conn.response :json, content_type: %r{application/json}
  end
  req.get('/json') do |conn|
    conn.body = body
  end
end

def thread_run
  threads = []
  10.times do
    threads << Thread.new do
      run.body.class
    end
  end
  threads.map(&:value)
end

pp thread_run
ksilex commented 2 years ago

olleolleolle

I'm sorry, it seems like problem can occure only when requesting ClickHouse database through http method, sometimes it parsed clickhouse's json response(returned Hash instance), sometimes returned it raw (as String instance) Also, it's never the case, when requests done successively (always returned Hash instance)

iMacTia commented 2 years ago

@ksilex sorry you're having this issue, could you please add the logger middleware to your stack and check if the failing responses have anything different from the others?

req = Faraday.new(config[:url]) do |conn|
  conn.response :logger, nil, { headers: true, bodies: true }
  conn.response :json, content_type: %r{application/json}
end

If you need, more info on the logger middleware are available here: https://lostisland.github.io/faraday/middleware/logger It's also worth pointing out that ClickHouse provide a database driver for Ruby, so I'm not sure if it makes sense to implement your own connector in this case. I'd suggest using the driver instead (there's even an ActiveRecord adapter)