SearchGoogleAdsStreamRequest is stuck without throwing error

[zubair-jaleel]

Describe the bug: For past few days I have a similar issue like this one: https://groups.google.com/g/adwords-api/c/Gx4QN7ER33s?pli=1. I need to get the campaign data from 2014 (backfill). I am using SearchGoogleAdsStreamRequest (search stream), and I am not using any pagination. I am using below python extraction code with 3 days difference (date interval) between the start & end date of the search query and I am running this extraction code in a loop (with 1 second sleep time between each loop iteration) to backfill data from 2014-01-01. After few loop iterations the SearchGoogleAdsStreamRequest is stuck, without any error/warning logs. I have rerun the script multiple times with different date intervals and sleep time intervals and even refreshed (reauthenticated) the GoogleAdsClient object for every 1000 request, but the script still got stuck at some point. When I ran the backfill for just few days its working fine. Therefore, the problem is only when I run the script for long time. Please let me know how to solve this.

This is my code:

import my_sql_queries

def _get_pmax_reports_for_client(self, ga_client: object, customer_id: int, start_date: str, end_date: str, retries: int = 0) -> customerType:

    try:

        campaign_data_request = ga_client.get_type("SearchGoogleAdsStreamRequest")
        campaign_data_request.customer_id = str(customer_id)

        type_campaign_status_enum = ga_client.get_type('CampaignStatusEnum')
        type_advertising_channel_enum = ga_client.get_type('AdvertisingChannelTypeEnum')
        type_advertising_channel_sub_enum = ga_client.get_type('AdvertisingChannelSubTypeEnum')
        type_bidding_strategy_enum = ga_client.get_type('BiddingStrategyTypeEnum')
        campaign_data_request.query = my_sql_queries.get_pmax_reports_for_client_query.format(start_date=start_date,
                                                                                           end_date=end_date)
        ga_service = ga_client.get_service("GoogleAdsService", version='v13')
        campaign_data_stream = ga_service.search_stream(campaign_data_request)

        pmax_campaign_data_list = []
        for batch in campaign_data_stream:
            for row in batch.results:
                row_list = {
                    'ROW_ID': row.segments.date + '_' + str([row.campaign.id](http://row.campaign.id/)),
                    'REPORT_DATE_LOC': row.segments.date,
                    'CAMPAIGN_ID': [row.campaign.id](http://row.campaign.id/),
                    'CAMPAIGN_NAME': [row.campaign.name](http://row.campaign.name/),
                    'BASE_CAMPAIGN_ID': re.sub('.*campaigns/', '', row.campaign.base_campaign),
                    'CAMPAIGN_STATUS': str([type_campaign_status_enum.CampaignStatus.Name](http://type_campaign_status_enum.campaignstatus.name/)(
                        row.campaign.status)).lower(),
                    'CAMPAIGN_TYPE': str([type_advertising_channel_enum.AdvertisingChannelType.Name](http://type_advertising_channel_enum.advertisingchanneltype.name/)(
                        row.campaign.advertising_channel_type)).lower(),
                    'CAMPAIGN_SUB_TYPE': str([type_advertising_channel_sub_enum.AdvertisingChannelSubType.Name](http://type_advertising_channel_sub_enum.advertisingchannelsubtype.name/)(
                        row.campaign.advertising_channel_sub_type)).lower(),
                    'BIDDING_STRATEGY_TYPE': str([type_bidding_strategy_enum.BiddingStrategyType.Name](http://type_bidding_strategy_enum.biddingstrategytype.name/)(
                        row.campaign.bidding_strategy_type)).lower(),
                    'CTR': row.metrics.ctr,
                    'VIEW_THROUGH_CONVERSIONS': row.metrics.view_through_conversions,
                    'CROSS_DEVICE_CONVERSIONS': row.metrics.cross_device_conversions,
                    'ENGAGEMENTS': row.metrics.engagements,
                    'VIDEO_VIEWS': row.metrics.video_views,
                    'ALL_CONVERSIONS': row.metrics.all_conversions,
                    'COST': row.metrics.cost_micros,
                    'COST_PER_CONVERSION': row.metrics.cost_per_conversion,
                    'CONVERSION_VALUE': row.metrics.conversions_value,
                    'PERCENT_NEW_VISITORS': row.metrics.percent_new_visitors,
                    'VIDEO_VIEW_RATE': row.metrics.video_view_rate,
                    'AVERAGE_TIME_ON_SITE': row.metrics.average_time_on_site,
                    'IMPRESSIONS': row.metrics.impressions,
                    'ENGAGEMENT_RATE': row.metrics.engagement_rate,
                    'EXTERNAL_CUSTOMER_ID': [row.customer.id](http://row.customer.id/),
                    'BOUNCE_RATE': row.metrics.bounce_rate,
                    'AVERAGE_PAGEVIEWS': row.metrics.average_page_views,
                    'ACCOUNT_DESCRIPTIVE_NAME': row.customer.descriptive_name,
                    'CONVERSION_RATE': row.metrics.conversions_from_interactions_rate,
                    'CONVERSIONS': row.metrics.conversions,
                    'INTERACTION_RATE': row.metrics.interaction_rate,
                    'INTERACTIONS': row.metrics.interactions,
                    'CLICKS': row.metrics.clicks,
                    'ACCOUNT_TIMEZONE_STANDARD': row.customer.time_zone,
                    'ACCOUNT_CURRENCY_CODE': row.customer.currency_code,
                    'CUSTOMER_ID': customer_id
                }
                pmax_campaign_data_list.append(row_list)

        return pmax_campaign_data_list

    except Exception as e:
        error_text = str(e)
        if ('429' in error_text or '500' in error_text or '503' in error_text) and retries <= 60:
            logging.error(f'Retrying after encountering Server Error. {error_text}')
            sleep(60)
            return self._get_pmax_reports_for_client(ga_client, customer_id, start_date, end_date,
                                                     retries=retries + 1)
        else:
            logging.error(f'Error occurred while getting data from Google Ads API (v13). {error_text}')
            raise

Steps to Reproduce:

• Run the above code in loop more than few hours

Expected behavior:

• Return campaign data for past few years

Client library version and API version:

• Client library version: google ads api python client version 21.1.0
• Google Ads API version: v13
• Python version: 3.11.3

  requirements.txt file content:

  google-ads==21.1.0
  boto3==1.26.120
  botocore==1.29.121
  gzip-stream==1.2.0
  snowflake-connector-python==3.0.3
  pytz==2023.3

Request/Response Logs:

AWS CloudWatch log of the request that I made:

INFO:logging_interceptor.py: Request made: ClientCustomerId: XXX, Host: googleads.googleapis.com, Method: /google.ads.googleads.v13.services.GoogleAdsService/SearchStream, RequestId: IokzYJhE1Z-nXZcin0q_ng, IsFault: False, FaultMessage: None

Request with header:

Method: /google.ads.googleads.v13.services.GoogleAdsService/SearchStream
Host: googleads.googleapis.com
Headers: {
  "developer-token": "REDACTED",
  "login-customer-id": "XXX",
  "x-goog-api-client": "gl-python/3.11.3 grpc/1.54.2 gax/2.11.0 gccl/21.1.0 pb/4.23.1",
  "x-goog-request-params": "customer_id=XXX"
}
Request: customer_id: "XXX"
query: "SELECT segments.date,  metrics.ctr, campaign.status, campaign.base_campaign,  campaign.name, metrics.view_through_conversions,  metrics.cross_device_conversions,  metrics.engagements,  metrics.video_views,  metrics.all_conversions,  metrics.cost_micros,  campaign.id,  metrics.cost_per_conversion,  metrics.conversions_value,  metrics.percent_new_visitors,  metrics.video_view_rate,  metrics.average_time_on_site,  metrics.impressions,  metrics.engagement_rate,  customer.id,  metrics.bounce_rate,  metrics.average_page_views,  customer.descriptive_name,  metrics.conversions_from_interactions_rate,  metrics.conversions,  metrics.interaction_rate,  metrics.interactions,  metrics.clicks,  customer.time_zone,  customer.currency_code,   campaign.advertising_channel_type, campaign.advertising_channel_sub_type, campaign.bidding_strategy_type FROM campaign WHERE segments.date >= \'2014-05-17\' AND segments.date <= \'2014-05-19\'"

Got No response for the request since my request is stuck without any warnings/errors

Anything else we should know about your project / environment:

• I have standard level Google Ads API access. I am running this python extraction script inside AWS ECS (docker container)
• I have already raised this issue with Google Ads Support team, they mailed me privately saying the request is working fine from their end and I have to contact the developer who maintain the python client library

[rhani96]

I am facing the same issue

[BenRKarl]

@zubair-jaleel @rhani96 apologies for the delayed response here. Are you still running into this problem? If so my recommendation is to reduce the size of your request so that the stream doesn't need to stay open as long. You could try reducing the date range from three days down to one.