Shopify / statsd-instrument

A StatsD client for Ruby apps. Provides metaprogramming methods to inject StatsD instrumentation into your code.
http://shopify.github.io/statsd-instrument
MIT License
574 stars 97 forks source link

Some metrics are being skipped on rescuing Timeout Error #310

Open AkshayGoyal022 opened 2 years ago

AkshayGoyal022 commented 2 years ago

I have an application which supports background jobs (report generations). I have a use-case in which if a report is taking too long i timeout the report. Below is the pseudo code:

def generate_report(user_id)
  timeout_after = 10800 # In seconds # 3hours
  StatsD.increment('report_generation', 1, tags:["name:report_generation_start", "user_id:#{user_id}"])
  formats = [:csv, :xlsx]
  formats.each do |r_format|
    begin
      Timeout.timeout(timeout_after) do
        report = send("generate_#{r_format}")
      end
    rescue Timeout::Error => e
      StatsD.increment('report_generation', 1, tags:["name:report_generation_timeout_error", "user_id:#{user_id}"])
      break
    end
  end
  StatsD.increment('report_generation', 1, tags:["name:report_generation_completed", "user_id:#{user_id}"])
end

I ran 3 background jobs for different users (let's say id 1, 2, 3).

On successful generation i see all 3 metrics being pushed correctly. However, on timeout error i see name:report_generation_start being pushed 3 times but name:report_generation_timeout_error, name:report_generation_completed being pushed only once or twice (there's always a missing entry atleast)

casperisfine commented 2 years ago

Would you be able to produce a reproduction script?

rafaelzimmermann commented 1 year ago

I can't reproduce it, I think I need more info. But the error described might happen when the error being thrown is not a Timeout::Error.

kleinkk76 commented 10 months ago

Duplicate of #