search

keolo / mixpanel_client

Ruby interface to the Mixpanel Data API
MIT License
148 stars 72 forks source link

Help with parallel request #60

Closed rderoldan1 closed 6 years ago

rderoldan1 commented 7 years ago

Any idea in how to handle Mixpanel::HTTPError: too many export requests in progress for this project while making parallel requests?

I'm using the last version of mix panel_client

Backtrace

    from /Users/ruben/.rvm/gems/ruby-1.9.3-p547/gems/mixpanel_client-4.1.6/lib/mixpanel/client.rb:130:in `block in prepare_parallel_request'
    from /Users/ruben/.rvm/gems/ruby-1.9.3-p547/gems/typhoeus-0.6.7/lib/typhoeus/request/callbacks.rb:129:in `call'
    from /Users/ruben/.rvm/gems/ruby-1.9.3-p547/gems/typhoeus-0.6.7/lib/typhoeus/request/callbacks.rb:129:in `block in execute_callbacks'
    from /Users/ruben/.rvm/gems/ruby-1.9.3-p547/gems/typhoeus-0.6.7/lib/typhoeus/request/callbacks.rb:128:in `map'
    from /Users/ruben/.rvm/gems/ruby-1.9.3-p547/gems/typhoeus-0.6.7/lib/typhoeus/request/callbacks.rb:128:in `execute_callbacks'
    from /Users/ruben/.rvm/gems/ruby-1.9.3-p547/gems/typhoeus-0.6.7/lib/typhoeus/request/operations.rb:35:in `finish'
    from /Users/ruben/.rvm/gems/ruby-1.9.3-p547/gems/typhoeus-0.6.7/lib/typhoeus/easy_factory.rb:105:in `block in set_callback'
    from /Users/ruben/.rvm/gems/ruby-1.9.3-p547/gems/ethon-0.6.3/lib/ethon/easy/response_callbacks.rb:65:in `call'
    from /Users/ruben/.rvm/gems/ruby-1.9.3-p547/gems/ethon-0.6.3/lib/ethon/easy/response_callbacks.rb:65:in `block in complete'
    from /Users/ruben/.rvm/gems/ruby-1.9.3-p547/gems/ethon-0.6.3/lib/ethon/easy/response_callbacks.rb:65:in `each'
    from /Users/ruben/.rvm/gems/ruby-1.9.3-p547/gems/ethon-0.6.3/lib/ethon/easy/response_callbacks.rb:65:in `complete'
    from /Users/ruben/.rvm/gems/ruby-1.9.3-p547/gems/ethon-0.6.3/lib/ethon/multi/operations.rb:148:in `check'
    from /Users/ruben/.rvm/gems/ruby-1.9.3-p547/gems/ethon-0.6.3/lib/ethon/multi/operations.rb:161:in `run'
    from /Users/ruben/.rvm/gems/ruby-1.9.3-p547/gems/ethon-0.6.3/lib/ethon/multi/operations.rb:43:in `perform'
    from /Users/ruben/.rvm/gems/ruby-1.9.3-p547/gems/typhoeus-0.6.7/lib/typhoeus/hydra/runnable.rb:21:in `run'
    from /Users/ruben/.rvm/gems/ruby-1.9.3-p547/gems/typhoeus-0.6.7/lib/typhoeus/hydra/memoizable.rb:51:in `run'
    from /Users/ruben/.rvm/gems/ruby-1.9.3-p547/gems/mixpanel_client-4.1.6/lib/mixpanel/client.rb:139:in `run_parallel_requests'
keolo commented 7 years ago

Sounds like you're hitting a request threshold from the Mixpanel Data API. Maybe you could throttle the number of requests you're sending at a time or use a background job with exponential backoff (to automatically retry failed requests)?