Click Performance Report API endpoint failing with a Zlib buffer Error

[saifmahamood]

Hi,

We have a Production job failing right now for the past couple of months due to a Ruby client error. I am trying to make this request to query the Adwords API for Click Performance Report.

This is the AWQL query I am trying to execute:

{:selector=>
  {:fields=>
    ["AdFormat", "AdGroupId", "AdNetworkType1", "CampaignId", "ClickType", "CreativeId", "CriteriaId", "CriteriaParameters", "Date", "Device", "GclId", "Page", "Slot"],
   :date_range=>{:min=>"20210211", :max=>"20210211"}},
 :report_name=>"AdWords Performance Report",
 :report_type=>"CLICK_PERFORMANCE_REPORT",
 :download_format=>"CSV",
 :date_range_type=>"CUSTOM_DATE"}

The log below are the error logs I am getting:

Longboat::API::PageFetcherCrashed: Pager fetcher crashed on interval #<Longboat::Extractors::Adwords::AdwordsPaginator:0x6f53d4a>: buffer error
/Users/saifmahamood/src/github.com/Shopify/longboat/lib/longboat/api/payload_enumerator.rb:28:in `block in build'
/Users/saifmahamood/src/github.com/Shopify/longboat/lib/longboat/api/payload_processor.rb:29:in `run'
/Users/saifmahamood/src/github.com/Shopify/longboat/lib/longboat/extractors/adwords/report_extractor.rb:44:in `block in extract'
/Users/saifmahamood/src/github.com/Shopify/longboat/lib/longboat/extractors/adwords/report_extractor.rb:17:in `block in extract'
/Users/saifmahamood/src/github.com/Shopify/longboat/lib/longboat/extractors/adwords/report_extractor.rb:16:in `extract'
/Users/saifmahamood/src/github.com/Shopify/longboat/lib/longboat/extractor.rb:385:in `block in extract_pipeline'
/Users/saifmahamood/.gem/jruby/2.5.3/gems/statsd-instrument-2.1.0/lib/statsd/instrument.rb:284:in `block in measure'
/Users/saifmahamood/.gem/jruby/2.5.3/gems/statsd-instrument-2.1.0/lib/statsd/instrument.rb:53:in `duration'
/Users/saifmahamood/.gem/jruby/2.5.3/gems/statsd-instrument-2.1.0/lib/statsd/instrument.rb:284:in `measure'
/Users/saifmahamood/src/github.com/Shopify/longboat/lib/longboat/extractor.rb:384:in `extract_pipeline'
/Users/saifmahamood/src/github.com/Shopify/longboat/lib/longboat/extractor.rb:568:in `block in run'
/Users/saifmahamood/.gem/jruby/2.5.3/gems/statsd-instrument-2.1.0/lib/statsd/instrument.rb:284:in `block in measure'
/Users/saifmahamood/.gem/jruby/2.5.3/gems/statsd-instrument-2.1.0/lib/statsd/instrument.rb:53:in `duration'
/Users/saifmahamood/.gem/jruby/2.5.3/gems/statsd-instrument-2.1.0/lib/statsd/instrument.rb:284:in `measure'
/Users/saifmahamood/src/github.com/Shopify/longboat/lib/longboat/extractor.rb:565:in `run'
/Users/saifmahamood/src/github.com/Shopify/longboat/lib/longboat.rb:125:in `run_job'
/Users/saifmahamood/src/github.com/Shopify/longboat/lib/longboat.rb:148:in `run'
/Users/saifmahamood/src/github.com/Shopify/longboat/Rakefile:30:in `block in <main>'
/Users/saifmahamood/.gem/jruby/2.5.3/gems/rake-12.3.2/exe/rake:27:in `<main>'
Caused by:
Zlib::BufError: buffer 
error org/jruby/ext/zlib/ZStream.java:135:in `finish'

This issue is not replicated with the PHP library client so we can rule out that this is an Adwords API issue. I would appreciate some help on this since our production jobs are stuck.

[mcloonan]

From that stack trace it doesn't look like you're actually using this client library. Can you provide some kind of code to reproduce that shows how you're invoking the call through our library?

[saifmahamood]

@mcloonan This is how we're invoking the call through the ruby library:

report_utils = adwords_authentication.report_utils(:v201809)

Log.info("Downloading report for #{@adwords_authentication.customer_id} on #{interval.start_date}")

csv_payload = report_utils.download_report(interval.report_definition)

interval.report_definition is

{:selector=>
  {:fields=>
    ["AdFormat", "AdGroupId", "AdNetworkType1", "CampaignId", "ClickType", "CreativeId", "CriteriaId", "CriteriaParameters", "Date", "Device", "GclId", "Page", "Slot"],
   :date_range=>{:min=>"20210211", :max=>"20210211"}},
 :report_name=>"AdWords Performance Report",
 :report_type=>"CLICK_PERFORMANCE_REPORT",
 :download_format=>"CSV",
 :date_range_type=>"CUSTOM_DATE"}

[saifmahamood]

Production stack trace:

[2021-02-18 00:08:41,537] {pod_launcher.py:136} INFO - Caused by:
[2021-02-18 00:08:41,538] {pod_launcher.py:136} INFO - Zlib::BufError: buffer error
[2021-02-18 00:08:41,538] {pod_launcher.py:136} INFO - org/jruby/ext/zlib/ZStream.java:135:in `finish'
[2021-02-18 00:08:41,538] {pod_launcher.py:136} INFO - /artifacts/jruby/2.5.0/gems/httpi-2.4.2/lib/httpi/adapter/net_http.rb:70:in `perform'
[2021-02-18 00:08:41,538] {pod_launcher.py:136} INFO - /artifacts/jruby/2.5.0/gems/httpi-2.4.2/lib/httpi/adapter/net_http.rb:47:in `block in request'
[2021-02-18 00:08:41,538] {pod_launcher.py:136} INFO - /artifacts/jruby/2.5.0/gems/httpi-2.4.2/lib/httpi/adapter/net_http.rb:87:in `block in do_request'
[2021-02-18 00:08:41,538] {pod_launcher.py:136} INFO - /artifacts/jruby/2.5.0/gems/httpi-2.4.2/lib/httpi/adapter/net_http.rb:85:in `do_request'
[2021-02-18 00:08:41,538] {pod_launcher.py:136} INFO - /artifacts/jruby/2.5.0/gems/httpi-2.4.2/lib/httpi/adapter/net_http.rb:38:in `request'
[2021-02-18 00:08:41,538] {pod_launcher.py:136} INFO - /artifacts/jruby/2.5.0/gems/httpi-2.4.2/lib/httpi.rb:161:in `request'
[2021-02-18 00:08:41,538] {pod_launcher.py:136} INFO - /artifacts/jruby/2.5.0/gems/httpi-2.4.2/lib/httpi.rb:133:in `post'
[2021-02-18 00:08:41,538] {pod_launcher.py:136} INFO - /artifacts/jruby/2.5.0/gems/google-ads-common-1.0.2/lib/ads_common/http.rb:59:in `post_response'
[2021-02-18 00:08:41,538] {pod_launcher.py:136} INFO - /artifacts/jruby/2.5.0/gems/google-adwords-api-1.4.0/lib/adwords_api/report_utils.rb:231:in `make_adhoc_request'
[2021-02-18 00:08:41,538] {pod_launcher.py:136} INFO - /artifacts/jruby/2.5.0/gems/google-adwords-api-1.4.0/lib/adwords_api/report_utils.rb:209:in `get_report_response'
[2021-02-18 00:08:41,538] {pod_launcher.py:136} INFO - /artifacts/jruby/2.5.0/gems/google-adwords-api-1.4.0/lib/adwords_api/report_utils.rb:56:in `download_report'
[2021-02-18 00:08:41,539] {pod_launcher.py:136} INFO - /app/lib/longboat/extractors/adwords/adwords_page_fetcher.rb:17:in `fetch'
[2021-02-18 00:08:41,539] {pod_launcher.py:136} INFO - /app/lib/longboat/api/payload_enumerator.rb:23:in `block in build'
[2021-02-18 00:08:41,539] {pod_launcher.py:136} INFO - /app/lib/longboat/api/payload_processor.rb:29:in `run'
[2021-02-18 00:08:41,539] {pod_launcher.py:136} INFO - /app/lib/longboat/extractors/adwords/report_extractor.rb:39:in `block in extract'
[2021-02-18 00:08:41,539] {pod_launcher.py:136} INFO - /app/lib/longboat/extractors/adwords/report_extractor.rb:17:in `block in extract'
[2021-02-18 00:08:41,539] {pod_launcher.py:136} INFO - /app/lib/longboat/extractors/adwords/report_extractor.rb:16:in `extract'
[2021-02-18 00:08:41,539] {pod_launcher.py:136} INFO - /app/lib/longboat/extractor.rb:385:in `block in extract_pipeline'
[2021-02-18 00:08:41,539] {pod_launcher.py:136} INFO - /artifacts/jruby/2.5.0/gems/statsd-instrument-2.1.0/lib/statsd/instrument.rb:284:in `block in measure'
[2021-02-18 00:08:41,539] {pod_launcher.py:136} INFO - /artifacts/jruby/2.5.0/gems/statsd-instrument-2.1.0/lib/statsd/instrument.rb:53:in `duration'
[2021-02-18 00:08:41,539] {pod_launcher.py:136} INFO - /artifacts/jruby/2.5.0/gems/statsd-instrument-2.1.0/lib/statsd/instrument.rb:284:in `measure'
[2021-02-18 00:08:41,539] {pod_launcher.py:136} INFO - /app/lib/longboat/extractor.rb:384:in `extract_pipeline'
[2021-02-18 00:08:41,539] {pod_launcher.py:136} INFO - /app/lib/longboat/extractor.rb:568:in `block in run'
[2021-02-18 00:08:41,539] {pod_launcher.py:136} INFO - /artifacts/jruby/2.5.0/gems/statsd-instrument-2.1.0/lib/statsd/instrument.rb:284:in `block in measure'
[2021-02-18 00:08:41,540] {pod_launcher.py:136} INFO - /artifacts/jruby/2.5.0/gems/statsd-instrument-2.1.0/lib/statsd/instrument.rb:53:in `duration'
[2021-02-18 00:08:41,540] {pod_launcher.py:136} INFO - /artifacts/jruby/2.5.0/gems/statsd-instrument-2.1.0/lib/statsd/instrument.rb:284:in `measure'
[2021-02-18 00:08:41,540] {pod_launcher.py:136} INFO - /app/lib/longboat/extractor.rb:565:in `run'
[2021-02-18 00:08:41,540] {pod_launcher.py:136} INFO - /app/lib/longboat.rb:125:in `run_job'
[2021-02-18 00:08:41,540] {pod_launcher.py:136} INFO - /app/lib/longboat.rb:89:in `run_with_manifest'
[2021-02-18 00:08:41,540] {pod_launcher.py:136} INFO - /app/Rakefile:36:in `block in <main>'
[2021-02-18 00:08:41,540] {pod_launcher.py:136} INFO - /artifacts/jruby/2.5.0/gems/rake-12.3.2/exe/rake:27:in `<main>'

[mcloonan]

It looks like it's some kind of issue with JRuby's interaction with HTTP requests. Neither our HTTP library nor our library directly uses "zlib" so I'm not really familiar with what might be going wrong here.

I think the easiest way to move forward would actually be to use the new Google Ads API client library instead of this one: https://github.com/googleads/google-ads-ruby

The new API is nearly feature complete with the AdWords API (including this use case) and is fully compatible with this client library (meaning you can use both in the same process without issue during your migration). Additionally, the AdWords API will be shut down at some point in the future, while we plan to support the Google Ads API going forward. So any work done to get this working on the AdWords API would have to be duplicated in the future to move to the Google Ads API.

If you do switch to that library, you could use a query on the click_view resource to fetch equivalent information, and since we're actively developing that library any issues would be a higher priority on our side.

[xaptronic]

Hello, I've taken over this issue on our side. The stack trace as you mention is not anything to do with the Google Adwords library, but rather is a "side effect" of limitations with JRuby and how our application was processing the result of the API call.

I have worked around this in our application. You can close this issue.

Thanks