Timeout for underlying library

[lpereirafernandez]

Hi there,

We have started to notice huge latencies when querying the GEO_PERFORMANCE_REPORT for certain accounts of our customers with significant activity / data.

We are using the Java SDK on version 4.1.0 (v201809) and in order to avoid timeouts we have increased the property ReportDownloadedTimeout to 6 minutes, as some of the queries for certain ad account can take even up to 5 minutes.

Based on our tests and taking a closer look to the error logs, we noticed for these "large accounts" that sometimes (exceptionally) the request are served in less than 30 seconds, so there is no problem. However, when they take more than 30 seconds we see the following SocketTimeoutException when reading:

INFO: Exception Message: Problem sending data to report download server.
com.google.api.ads.adwords.lib.utils.ReportException: Problem sending data to report download server.
    at com.google.api.ads.adwords.lib.utils.AdHocReportDownloadHelperImpl.downloadReport(AdHocReportDownloadHelperImpl.java:91)
    at com.google.api.ads.adwords.lib.utils.AdHocReportDownloadHelperImpl.downloadReport(AdHocReportDownloadHelperImpl.java:60)
    at sun.reflect.GeneratedMethodAccessor1179.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.google.api.ads.common.lib.utils.AdsUtilityInvocationHandler.handleInvocation(AdsUtilityInvocationHandler.java:46)
    at com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:84)
    at com.sun.proxy.$Proxy217.downloadReport(Unknown Source)
    at com.google.api.ads.adwords.lib.utils.AdHocReportDownloadHelper.downloadReport(AdHocReportDownloadHelper.java:46)
    at com.google.api.ads.adwords.lib.utils.v201809.ReportDownloader.downloadReport(ReportDownloader.java:65)
    at com.reportingninja.server.ComponentExecutorAdwords.runReport(ComponentExecutorAdwords.java:3018)
    at com.reportingninja.server.ComponentExecutorAdwords.executeBasicComponent(ComponentExecutorAdwords.java:135)
    at com.reportingninja.server.ComponentExecutor.executeAdwordsComponent(ComponentExecutor.java:179)
    at com.reportingninja.view.advanced.ComponentViewer.viewAdwordsComponent(ComponentViewer.java:194)
    at com.reportingninja.view.advanced.ComponentViewer.viewComponent(ComponentViewer.java:63)
    at com.reportingninja.view.menu.reports.components.ReportComponent$8.run(ReportComponent.java:1306)
Caused by: java.net.SocketTimeoutException: Read timed out
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
    at java.net.SocketInputStream.read(SocketInputStream.java:171)
    at java.net.SocketInputStream.read(SocketInputStream.java:141)
    at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
    at sun.security.ssl.InputRecord.read(InputRecord.java:503)
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:983)
    at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:940)
    at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
    at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
    at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
    at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
    at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:735)
    at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1587)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492)
    at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
    at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:347)
    at com.google.api.client.http.javanet.NetHttpResponse.<init>(NetHttpResponse.java:37)
    at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:94)
    at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:972)
    at com.google.api.ads.adwords.lib.utils.AdHocReportDownloadHelperImpl.downloadReport(AdHocReportDownloadHelperImpl.java:75)
    ... 15 more

Then it seems like the connection is reset and a new attempt takes place, but again, if it takes longer than 30 seconds, we see another time the same exception, and so on. As commented before, we have waited for more than 5 minutes to see the request completed in less than 30 seconds after the last timeout. Some other times we received exceeded rate exceptions.

It looks to us that com.google.api.client.http.HttpRequest has a 30 seconds read timeout we can't adjust, and when the AdWords API backend takes longer than 30 seconds to serve the request, an exception is raised and the process starts again.

Not sure if this makes any sense for you. Is there any way to skip or to increase that 30 seconds read timeout for the underlying library?

Many thanks in advance, Luis

[lpereirafernandez]

Hi again,

We figured out part of the problem. While we were setting ReportDownloadedTimeout to 6 minutes when creating the ReportingConfiguration instance for the AdWordsSession.Builder(), when creating the instance for the ReportDownloader we were adjusting by mistake the ReportDownloadTimeout to 30 seconds. Our fault (at least in part of it).

That part is solved / clear now, but we are still seeing significant response times (up to 3 minutes this morning) when querying the GEO_PERFORMANCE_REPORT for certain accounts of our customers with significant activity / data. The strange thing is that this query is just one of many queries in a report for the same ad account, including multiple levels like account, campaign, ad group, ads, keywords and some other. For the rest of levels the queries are served almost instantaneously, and it only gets stuck for GEO_PERFORMANCE_REPORT, even though in the result set there is only one country.

The query is really simple:

<reportDefinition xmlns="https://adwords.google.com/api/adwords/cm/v201809">
<selector>
<fields>Country/Territory</fields>
<fields>Impressions</fields>
<fields>Clicks</fields>
<fields>Cost</fields>
</selector>
<reportName>GEO_PERFORMANCE_REPORT_1548320781068</reportName>
<reportType>GEO_PERFORMANCE_REPORT</reportType>
<dateRangeType>LAST_MONTH</dateRangeType>
<downloadFormat>CSV</downloadFormat>
</reportDefinition>

I could give it a try with customer id 8288984184. Not sure if this is specific for the Java SDK or just generic for the AdWords API backend.

Many thanks, Luis

[nwbirnie]

Glad to hear you got this fixed. Would you kindly open the latency issue on the forum please?