googleads / google-ads-python

Google Ads API Client Library for Python
Apache License 2.0
515 stars 480 forks source link

Unable to create sitelinks for ad group in bulk (batch job) #386

Closed pySilver closed 3 years ago

pySilver commented 3 years ago

UPDATE: Partially solved. Please see following comment.

Hi! Is that possible to create sitelinks and assign them into ad group in the same Batch Job?

I have a method that creates a sitelink extension like this:

    def _get_extension_feed_item_resource_name(self, feed_item_id: str) -> str:
        """
        Returns resource_name for Extension Feed Item by ID

        :param feed_item_id: Feed item ID
        :return: Feed item resource name
        """
        extension_feed_item_service = self.client.get_service(
            "ExtensionFeedItemService", version=GOOGLE_ADS_API_VERSION
        )
        return extension_feed_item_service.extension_feed_item_path(self.customer_id, feed_item_id)

    def _build_ad_group_sitelink_extension_operation(
        self, ad_group_resource_name: str, link_text: str, final_url: str
    ) -> ExtensionFeedItemOperation:
        """
        Helper method that creates sitelink extension feed item for ad group.

        :param ad_group_resource_name: Ad Group resource name
        :param link_text: Link text
        :param final_url: Target URL
        :return: ExtensionFeedItemOperation
        """
        extension_type_enum = self.client.get_type("ExtensionTypeEnum")
        extension_feed_item_operation = self.client.get_type(
            "ExtensionFeedItemOperation", version=GOOGLE_ADS_API_VERSION
        )
        extension_feed_item = extension_feed_item_operation.create
        extension_feed_item.extension_type = extension_type_enum.SITELINK
        extension_feed_item.sitelink_feed_item.link_text = link_text
        extension_feed_item.targeted_ad_group = ad_group_resource_name
        extension_feed_item.sitelink_feed_item.final_urls.append(final_url)
        # Creates a resource name using the temporary ID.
        feed_item_id = str(self._get_next_temporary_id())
        extension_feed_item.resource_name = self._get_extension_feed_item_resource_name(feed_item_id)

        return extension_feed_item_operation

So I'm creating few sitelinks using _build_ad_group_sitelink_extension_operation method and then I'm trying to assign them into ad group using

# looping through link items ...
extension_feed_item_operation = self._build_ad_group_sitelink_extension_operation(
                    ad_group_resource_name=ad_group_resource_name,
                    link_text=item["text"],
                    final_url=item["url"],
                )
link_resource_id = extension_feed_item_operation.create.resource_name
# memorizing link_resource_id for AdGroupExtensionSettingOperation

Feed item resource names looks the following:

Out[19]:
['customers/<hidden>/extensionFeedItems/-10',
 'customers/<hidden>/extensionFeedItems/-18',
 'customers/<hidden>/extensionFeedItems/-26',
 'customers/<hidden>/extensionFeedItems/-34']

and then I'm creating AdGroupExtensionSettingOperation

    def _build_ad_group_extension_settings_operation(
        self, ad_group_resource_name: str, feed_item_resource_names: List[str]
    ) -> AdGroupExtensionSettingOperation:
        """
        Builds Ad Group extension setting operation
        :param ad_group_resource_name: Ad Group resource name
        :return: AdGroupExtensionSettingOperation
        """
        ad_group_ext_setting_operation = self.client.get_type(
            "AdGroupExtensionSettingOperation", version=GOOGLE_ADS_API_VERSION
        )
        extension_type_enum = self.client.get_type("ExtensionTypeEnum", version=GOOGLE_ADS_API_VERSION)

        ad_group_ext_setting = ad_group_ext_setting_operation.create
        ad_group_ext_setting.ad_group = ad_group_resource_name
        ad_group_ext_setting.extension_type = extension_type_enum.SITELINK
        ad_group_ext_setting.extension_feed_items.extend(feed_item_resource_names)
        return ad_group_ext_setting_operation

finally I'm appending my ExtensionFeedItemOperations and AdGroupExtensionSettingOperation to a batch job;

In [35]: data['operations']['ad_group_links']
Out[35]:
[extension_feed_item_operation {
   create {
     resource_name: "customers/<hidden>/extensionFeedItems/-10"
     sitelink_feed_item {
       link_text: "Buty zimowe"
       final_urls: "https://domaincom/pl/buty-zimowe-damskie/2a/"
     }
     extension_type: SITELINK
     targeted_ad_group: "customers/<hidden>/adGroups/-3"
   }
 },
 ad_group_extension_setting_operation {
   create {
     extension_type: SITELINK
     ad_group: "customers/<hidden>/adGroups/-3"
     extension_feed_items: "customers/<hidden>/extensionFeedItems/-10"
   }
 },
 extension_feed_item_operation {
   create {
     resource_name: "customers/<hidden>/extensionFeedItems/-18"
     sitelink_feed_item {
       link_text: "Buty zimowe"
       final_urls: "https://domaincom/pl/buty-zimowe-damskie/2a__zapiecie-buta-zamek/"
     }
     extension_type: SITELINK
     targeted_ad_group: "customers/<hidden>/adGroups/-11"
   }
 },
 ad_group_extension_setting_operation {
   create {
     extension_type: SITELINK
     ad_group: "customers/<hidden>/adGroups/-11"
     extension_feed_items: "customers/<hidden>/extensionFeedItems/-18"
   }
 },
 extension_feed_item_operation {
   create {
     resource_name: "customers/<hidden>/extensionFeedItems/-26"
     sitelink_feed_item {
       link_text: "Kozaki damskie"
       final_urls: "https://domaincom/pl/kozaki-damskie/2a/"
     }
     extension_type: SITELINK
     targeted_ad_group: "customers/<hidden>/adGroups/-19"
   }
 },
 ad_group_extension_setting_operation {
   create {
     extension_type: SITELINK
     ad_group: "customers/<hidden>/adGroups/-19"
     extension_feed_items: "customers/<hidden>/extensionFeedItems/-26"
   }
 },
 extension_feed_item_operation {
   create {
     resource_name: "customers/<hidden>/extensionFeedItems/-34"
     sitelink_feed_item {
       link_text: "Kozaki damskie"
       final_urls: "https://domaincom/pl/kozaki-damskie/2a__zapiecie-buta-zamek/"
     }
     extension_type: SITELINK
     targeted_ad_group: "customers/<hidden>/adGroups/-27"
   }
 },
 ad_group_extension_setting_operation {
   create {
     extension_type: SITELINK
     ad_group: "customers/<hidden>/adGroups/-27"
     extension_feed_items: "customers/<hidden>/extensionFeedItems/-34"
   }
 }]

The response from batch job contains the following error:

Batch job #50 has a status "Field 'resource_name' cannot be modified by 'CREATE' operation., at mutate_operations[50].extension_feed_item_operation.create.resource_name" and response type "N/A"
Batch job #51 has a status "Multiple errors in ‘details’. First error: Field must be set., at mutate_operations[51].ad_group_extension_setting_operation" and response type "N/A"

So it looks like I cannot use temporary IDs for feed items? What should I do differently to create feed items and assign into ad group in a single Batch Job then?

pySilver commented 3 years ago

Ok, I've managed to create sitelinks in batch job by NOT setting resource_name on extension_feed_item_operation but rather using Temporary ID for .id property of operation like this:

[extension_feed_item_operation {
   create {
     sitelink_feed_item {
       link_text: "Buty zimowe"
       final_urls: "https://domain.com/buty-zimowe-damskie/2a/"
     }
     extension_type: SITELINK
     id: -10
     targeted_ad_group: "customers/<hidden>/adGroups/-3"
   }
 },
 ad_group_extension_setting_operation {
   create {
     extension_type: SITELINK
     ad_group: "customers/<hidden>/adGroups/-3"
     extension_feed_items: "customers/<hidden>/extensionFeedItems/-10"
   }
 },
 extension_feed_item_operation {
   create {
     sitelink_feed_item {
       link_text: "Buty zimowe"
       final_urls: "https://domain.com/buty-zimowe-damskie/2a__zapiecie-buta-zamek/"
     }
     extension_type: SITELINK
     id: -18
     targeted_ad_group: "customers/<hidden>/adGroups/-11"
   }
 },
 ad_group_extension_setting_operation {
   create {
     extension_type: SITELINK
     ad_group: "customers/<hidden>/adGroups/-11"
     extension_feed_items: "customers/<hidden>/extensionFeedItems/-18"
   }
 },
 extension_feed_item_operation {
   create {
     sitelink_feed_item {
       link_text: "Kozaki damskie"
       final_urls: "https://domain.com/kozaki-damskie/2a/"
     }
     extension_type: SITELINK
     id: -26
     targeted_ad_group: "customers/<hidden>/adGroups/-19"
   }
 },
 ad_group_extension_setting_operation {
   create {
     extension_type: SITELINK
     ad_group: "customers/<hidden>/adGroups/-19"
     extension_feed_items: "customers/<hidden>/extensionFeedItems/-26"
   }
 },
 extension_feed_item_operation {
   create {
     sitelink_feed_item {
       link_text: "Kozaki damskie"
       final_urls: "https://domain.com/kozaki-damskie/2a__zapiecie-buta-zamek/"
     }
     extension_type: SITELINK
     id: -34
     targeted_ad_group: "customers/<hidden>/adGroups/-27"
   }
 },
 ad_group_extension_setting_operation {
   create {
     extension_type: SITELINK
     ad_group: "customers/<hidden>/adGroups/-27"
     extension_feed_items: "customers/<hidden>/extensionFeedItems/-34"
   }
 }]

However when reading errors I'm still seeing this error:

Batch job #51 has a status "Multiple errors in ‘details’. First error: Field must be set., at mutate_operations[51].ad_group_extension_setting_operation" and response type "N/A"

#....
Out[26]:
operation_index: 51
status {
  code: 3
  message: "Multiple errors in \342\200\230details\342\200\231. First error: Field must be set., at mutate_operations[51].ad_group_extension_setting_operation"
  details {
    type_url: "type.googleapis.com/google.ads.googleads.v6.errors.GoogleAdsFailure"
    value: "\n^\n\003\270\006$\022\022Field must be set.\032\002*\000\"?\022\025\n\021mutate_operations\0303\022&\n$ad_group_extension_setting_operation"
  }
}

What field I'm not setting? I'm setting all required fields from https://developers.google.com/google-ads/api/reference/rpc/v6/AdGroupExtensionSetting?hl=en like this:

    def _build_ad_group_extension_settings_operation(
        self, ad_group_resource_name: str, feed_item_resource_names: List[str]
    ) -> AdGroupExtensionSettingOperation:
        """
        Builds Ad Group extension setting operation
        :param ad_group_resource_name: Ad Group resource name
        :return: AdGroupExtensionSettingOperation
        """
        ad_group_ext_setting_operation = self.client.get_type(
            "AdGroupExtensionSettingOperation", version=GOOGLE_ADS_API_VERSION
        )
        extension_type_enum = self.client.get_type("ExtensionTypeEnum", version=GOOGLE_ADS_API_VERSION)

        ad_group_ext_setting = ad_group_ext_setting_operation.create
        ad_group_ext_setting.resource_name = ad_group_resource_name
        ad_group_ext_setting.ad_group = ad_group_resource_name
        ad_group_ext_setting.extension_type = extension_type_enum.SITELINK
        ad_group_ext_setting.extension_feed_items.extend(feed_item_resource_names)

        return ad_group_ext_setting_operation
BBalazsJurgen commented 3 years ago

Same issue here. In fact, I can confirm it's not only Sitelinks, but other extensions as well (my example uses Promotion Extensions).

Batch job #0 has a status "Field 'resource_name' cannot be modified by 'CREATE' operation., at mutate_operations[0].extension_feed_item_operation.create.resource_name" and response type "N/A"
Batch job #1 has a status "Multiple errors in ‘details’. First error: Field must be set., at mutate_operations[1].ad_group_extension_setting_operation" and response type "N/A"

I didn't try going with setting the .id property, but that should not be the solution as described in: https://developers.google.com/google-ads/api/docs/batch-processing/temporary-ids

The operations list looks like this:

[extension_feed_item_operation {
  create {
    resource_name: "customers/1234567890/extensionFeedItems/-2"
    promotion_feed_item {
      discount_modifier: UP_TO
      occasion: SUMMER_SALE
      promotion_target: "Heisenberg Goodies"
      percent_off: 17
      final_urls: "http://www.example.com/sale"
    }
    extension_type: PROMOTION
    targeted_ad_group: "customers/1234567890/adGroups/987654321098"
  }
}
, ad_group_extension_setting_operation {
  create {
    extension_type: PROMOTION
    ad_group: "customers/1234567890/adGroups/987654321098"
    extension_feed_items: "customers/1234567890/extensionFeedItems/-2"
  }
}
]

This should work, but it does not.

pySilver commented 3 years ago

@BBalazsJurgen I'm glad I'm not alone here.

BBalazsJurgen commented 3 years ago

Ok, so after playing around a little, I modified the original "Batch Job with Temporary IDs" example file (found here: https://github.com/googleads/google-ads-python/blob/a07a2493d4fbcb5064cb3bd51b636880f316b418/examples/campaign_management/add_complete_campaigns_using_batch_job.py) to upload callout extensions (because they're simple) instead of keywords, trying to change the original working example as little as possible.

Turns out the error message was exactly the same (both of them), so I have a guess that batch jobs don't (might not) support temporary IDs for ad extensions. I seriously hope someone can prove me otherwise. I don't think setting .id like you did would work, because it is an output only field. It might not produce an error, but I guess it's pretty much the same as not setting anything, you shouldn't be able to reference it with that temporary id later.

All ad extension examples use mass mutate operations without a Batch Job. Do you think the solution would be to rewrite everything to make tons of mutate requests instead of a batch job? I fear that would go up against the api limit of 5000 mutates per day pretty fast (that's basically 2500 extension creations and 2500 extension setting operations), and sounds jolly inefficient.

What did you find?

pySilver commented 3 years ago

@BBalazsJurgen can you please copy your comment here https://groups.google.com/u/1/g/adwords-api/c/wAZoM5d8e-I ?

pySilver commented 3 years ago

I don't think setting .id like you did would work, because it is an output only field. it works I see extension feed items created on Google Ads side this way without any errors. But Adding them (using their Temporary ID) fails with Field must be set. error.

As a temporary solution (might not be suitable for everyone) I'm keeping all created extensions on my end, so I can use real resource_name on a next Batch Job. But that sucks.

pySilver commented 3 years ago

Also you a probably wrong about limits. Its 5000 items limit per mutate call.

pySilver commented 3 years ago

@BenRKarl @AndrewMBurke guy's, sorry to ping you this way. Can you please take a look at this issue, please?

BBalazsJurgen commented 3 years ago

@pySilver You might be right about limits, I was never really bothered about them, especially since I always tried keeping to Batch Jobs wherever possible.

My idea was to upload (create) all ExtensionFeedItems on one batchjob, then query a list of all extensions, somehow identify the new items, get their resource names and use ExtensionSettingOperations on another batchjob. But that is remarkably tedious and prone to all sorts of errors.

Using simple mutate operations, as in the examples would probably work, but it would still be nice to use batch jobs and temporary ids. Making a mutate request, then getting the resource name from the response and sending another request to assign them just doesn't sound right.

BenRKarl commented 3 years ago

Hi All - apologies for the delayed response, I'm looking into this now. I see you've also started a thread with Support, which is great, because this issue is likely not a problem with the Python library here.

BenRKarl commented 3 years ago

Looking over the discussion here it seems as though the main problem is the error:

"Field 'resource_name' cannot be modified by 'CREATE' operation., at mutate_operations[50].extension_feed_item_operation.create.resource_name" and response type "N/A"

Since our documentation states that this is how temporary IDs are supposed to be used, this error doesn't make sense.

I'll dig around and see if there's something special about ExtensionFeedItems that's causing this. Have you all encountered this error with any other resource type?

BenRKarl commented 3 years ago

@pySilver in your first comment, the first object in your batch job is:

extension_feed_item_operation {
   create {
     resource_name: "customers/<hidden>/extensionFeedItems/-10"
     sitelink_feed_item {
       link_text: "Buty zimowe"
       final_urls: "https://domaincom/pl/buty-zimowe-damskie/2a/"
     }
     extension_type: SITELINK
     targeted_ad_group: "customers/<hidden>/adGroups/-3"
   }
 }

Are you creating the ad group with resource name "customers/<hidden>/adGroups/-3" earlier in that same request?

I'm trying to follow the chain of created entities to make sure the temporary IDs in resource names are accounted for and I don't see where that ad group is created. My sense is that this problem might be related to the temporary IDs in some way, and it's just returning a generic error.

pySilver commented 3 years ago

@BenRKarl thanks for looking into that!

Are you creating the ad group with resource name "customers//adGroups/-3" earlier in that same request?

Yes, I'm sure I'm doing that in the proper order:

  1. Budgets
  2. Campaigns
  3. Ad Groups
  4. Ads
  5. Extension Feed Items
  6. Ad Group Extension Settings

in other words, ad group operation was added before adding extension feed item referencing that ad group.

As I've mentioned earlier, if I'm setting temporary id into.id field no error is generated and I can see items in extension feed.

BenRKarl commented 3 years ago

Got it thank you - the id field is read-only so you shouldn't be setting it, using temporary IDs should be done via the resource_name field.

Do you happen to have a batch job ID from one of the failed requests I could use to investigate?

pySilver commented 3 years ago

Yeah, but for some reason .id works and resource_name not :) I just tried to solve my issue hehe.

I'm in a process of refactoring our integration (which is complex) hopefully I'll have Batch Job ID in a few hours that I can share with you.

BenRKarl commented 3 years ago

Great, thank you!

I would expect that setting the id field would give you an error similar to the one you're receiving for resource_name.

pySilver commented 3 years ago

@BenRKarl here it is customers/6830657154/batchJobs/4488130954

Batch Job result log:

In [15]: batch_job_results = ads.batch_job.service.list_batch_job_results(ads.batch_job.resource_name, page_size=1000)
    ...: for batch_job_result in batch_job_results:
    ...:     status = batch_job_result.status.message
    ...:     status = status if status else "N/A"
    ...:     result = batch_job_result.mutate_operation_response
    ...:     result = result if result.ByteSize() else "N/A"
    ...:     print(
    ...:         f"Batch job #{batch_job_result.operation_index} "
    ...:         f'has a status "{status}" and response type "{result}"'
    ...:     )
    ...:
Request made: ClientCustomerId: 6830657154, Host: googleads.googleapis.com:443, Method: /google.ads.googleads.v6.services.BatchJobService/ListBatchJobResults, RequestId: JpY6k2JF0OAxCpfZyDdL2g, IsFault: False, FaultMessage: None
Batch job #0 has a status "N/A" and response type "campaign_budget_result {
  resource_name: "customers/6830657154/campaignBudgets/8298106024"
}
"
Batch job #1 has a status "N/A" and response type "campaign_result {
  resource_name: "customers/6830657154/campaigns/12475670790"
}
"
Batch job #2 has a status "N/A" and response type "campaign_criterion_result {
  resource_name: "customers/6830657154/campaignCriteria/12475670790~2616"
}
"
Batch job #3 has a status "N/A" and response type "campaign_criterion_result {
  resource_name: "customers/6830657154/campaignCriteria/12475670790~30001"
}
"
Batch job #4 has a status "N/A" and response type "campaign_criterion_result {
  resource_name: "customers/6830657154/campaignCriteria/12475670790~30002"
}
"
Batch job #5 has a status "N/A" and response type "campaign_criterion_result {
  resource_name: "customers/6830657154/campaignCriteria/12475670790~30000"
}
"
Batch job #6 has a status "N/A" and response type "ad_group_result {
  resource_name: "customers/6830657154/adGroups/121584987754"
}
"
Batch job #7 has a status "N/A" and response type "ad_group_result {
  resource_name: "customers/6830657154/adGroups/121584987794"
}
"
Batch job #8 has a status "N/A" and response type "ad_group_result {
  resource_name: "customers/6830657154/adGroups/121584987834"
}
"
Batch job #9 has a status "N/A" and response type "ad_group_result {
  resource_name: "customers/6830657154/adGroups/121584987994"
}
"
Batch job #10 has a status "N/A" and response type "ad_group_result {
  resource_name: "customers/6830657154/adGroups/121584988034"
}
"
Batch job #11 has a status "N/A" and response type "ad_group_result {
  resource_name: "customers/6830657154/adGroups/121584988074"
}
"
Batch job #12 has a status "N/A" and response type "ad_group_criterion_result {
  resource_name: "customers/6830657154/adGroupCriteria/121584987754~1185937257293"
}
"
Batch job #13 has a status "N/A" and response type "ad_group_criterion_result {
  resource_name: "customers/6830657154/adGroupCriteria/121584987754~1185937256533"
}
"
Batch job #14 has a status "N/A" and response type "ad_group_criterion_result {
  resource_name: "customers/6830657154/adGroupCriteria/121584987754~1185937257093"
}
"
Batch job #15 has a status "N/A" and response type "ad_group_criterion_result {
  resource_name: "customers/6830657154/adGroupCriteria/121584987754~1185937256333"
}
"
Batch job #16 has a status "N/A" and response type "ad_group_criterion_result {
  resource_name: "customers/6830657154/adGroupCriteria/121584987794~1185937257253"
}
"
Batch job #17 has a status "N/A" and response type "ad_group_criterion_result {
  resource_name: "customers/6830657154/adGroupCriteria/121584987794~1185937256373"
}
"
Batch job #18 has a status "N/A" and response type "ad_group_criterion_result {
  resource_name: "customers/6830657154/adGroupCriteria/121584987794~1185937257333"
}
"
Batch job #19 has a status "N/A" and response type "ad_group_criterion_result {
  resource_name: "customers/6830657154/adGroupCriteria/121584987794~1185937256573"
}
"
Batch job #20 has a status "N/A" and response type "ad_group_criterion_result {
  resource_name: "customers/6830657154/adGroupCriteria/121584987834~1185937257493"
}
"
Batch job #21 has a status "N/A" and response type "ad_group_criterion_result {
  resource_name: "customers/6830657154/adGroupCriteria/121584987834~1185937256613"
}
"
Batch job #22 has a status "N/A" and response type "ad_group_criterion_result {
  resource_name: "customers/6830657154/adGroupCriteria/121584987834~1185937257573"
}
"
Batch job #23 has a status "N/A" and response type "ad_group_criterion_result {
  resource_name: "customers/6830657154/adGroupCriteria/121584987834~1185937256813"
}
"
Batch job #24 has a status "N/A" and response type "ad_group_criterion_result {
  resource_name: "customers/6830657154/adGroupCriteria/121584987994~1185937257533"
}
"
Batch job #25 has a status "N/A" and response type "ad_group_criterion_result {
  resource_name: "customers/6830657154/adGroupCriteria/121584987994~1185937256773"
}
"
Batch job #26 has a status "N/A" and response type "ad_group_criterion_result {
  resource_name: "customers/6830657154/adGroupCriteria/121584987994~1185937257733"
}
"
Batch job #27 has a status "N/A" and response type "ad_group_criterion_result {
  resource_name: "customers/6830657154/adGroupCriteria/121584987994~1185937256853"
}
"
Batch job #28 has a status "N/A" and response type "ad_group_criterion_result {
  resource_name: "customers/6830657154/adGroupCriteria/121584988034~1185937257773"
}
"
Batch job #29 has a status "N/A" and response type "ad_group_criterion_result {
  resource_name: "customers/6830657154/adGroupCriteria/121584988034~1185937257013"
}
"
Batch job #30 has a status "N/A" and response type "ad_group_criterion_result {
  resource_name: "customers/6830657154/adGroupCriteria/121584988074~1185937257813"
}
"
Batch job #31 has a status "N/A" and response type "ad_group_criterion_result {
  resource_name: "customers/6830657154/adGroupCriteria/121584988074~1185937257053"
}
"
Batch job #32 has a status "N/A" and response type "ad_group_ad_result {
  resource_name: "customers/6830657154/adGroupAds/121584987754~502679723970"
}
"
Batch job #33 has a status "N/A" and response type "ad_group_ad_result {
  resource_name: "customers/6830657154/adGroupAds/121584987754~502679723973"
}
"
Batch job #34 has a status "N/A" and response type "ad_group_ad_result {
  resource_name: "customers/6830657154/adGroupAds/121584987794~502679723976"
}
"
Batch job #35 has a status "N/A" and response type "ad_group_ad_result {
  resource_name: "customers/6830657154/adGroupAds/121584987794~502679723979"
}
"
Batch job #36 has a status "N/A" and response type "ad_group_ad_result {
  resource_name: "customers/6830657154/adGroupAds/121584987834~502679723982"
}
"
Batch job #37 has a status "N/A" and response type "ad_group_ad_result {
  resource_name: "customers/6830657154/adGroupAds/121584987834~502679723985"
}
"
Batch job #38 has a status "N/A" and response type "ad_group_ad_result {
  resource_name: "customers/6830657154/adGroupAds/121584987994~502679723988"
}
"
Batch job #39 has a status "N/A" and response type "ad_group_ad_result {
  resource_name: "customers/6830657154/adGroupAds/121584987994~502679723991"
}
"
Batch job #40 has a status "N/A" and response type "ad_group_ad_result {
  resource_name: "customers/6830657154/adGroupAds/121584988034~502679723994"
}
"
Batch job #41 has a status "N/A" and response type "ad_group_ad_result {
  resource_name: "customers/6830657154/adGroupAds/121584988034~502679723997"
}
"
Batch job #42 has a status "N/A" and response type "ad_group_ad_result {
  resource_name: "customers/6830657154/adGroupAds/121584988074~502679724120"
}
"
Batch job #43 has a status "N/A" and response type "ad_group_ad_result {
  resource_name: "customers/6830657154/adGroupAds/121584988074~502679724123"
}
"
Batch job #44 has a status "N/A" and response type "extension_feed_item_result {
  resource_name: "customers/6830657154/extensionFeedItems/165495370142"
}
"
Batch job #45 has a status "Multiple errors in ‘details’. First error: Field must be set., at mutate_operations[45].ad_group_extension_setting_operation" and response type "N/A"
Batch job #46 has a status "N/A" and response type "extension_feed_item_result {
  resource_name: "customers/6830657154/extensionFeedItems/165473800714"
}
"
Batch job #47 has a status "Multiple errors in ‘details’. First error: Field must be set., at mutate_operations[47].ad_group_extension_setting_operation" and response type "N/A"
Batch job #48 has a status "N/A" and response type "extension_feed_item_result {
  resource_name: "customers/6830657154/extensionFeedItems/165495385895"
}
"
Batch job #49 has a status "Multiple errors in ‘details’. First error: Field must be set., at mutate_operations[49].ad_group_extension_setting_operation" and response type "N/A"
Batch job #50 has a status "N/A" and response type "extension_feed_item_result {
  resource_name: "customers/6830657154/extensionFeedItems/165495389693"
}
"
Batch job #51 has a status "Multiple errors in ‘details’. First error: Field must be set., at mutate_operations[51].ad_group_extension_setting_operation" and response type "N/A"
pySilver commented 3 years ago

Funny fact. Once job is done, I dediced to check what report api says for query like this

query = """
        SELECT
            extension_feed_item.extension_type,
            extension_feed_item.id,
            extension_feed_item.resource_name,
            extension_feed_item.sitelink_feed_item.final_urls,
            extension_feed_item.sitelink_feed_item.line1,
            extension_feed_item.sitelink_feed_item.line2,
            extension_feed_item.sitelink_feed_item.link_text,
            extension_feed_item.status,
            extension_feed_item.targeted_ad_group,
            extension_feed_item.targeted_campaign
        FROM extension_feed_item
        """

and turns out my sitelinks are associated with ad groups:

[extension_feed_item {
   resource_name: "customers/6830657154/extensionFeedItems/163933065046"
   sitelink_feed_item {
     link_text: "Site"
     final_urls: "https://domain.com/pl/"
   }
   status: REMOVED
   extension_type: SITELINK
   id: 163933065046
 },
 extension_feed_item {
   resource_name: "customers/6830657154/extensionFeedItems/163933065049"
   sitelink_feed_item {
     link_text: "Site"
     final_urls: "https://domain.com/pl/"
   }
   status: REMOVED
   extension_type: SITELINK
   id: 163933065049
 },
 extension_feed_item {
   resource_name: "customers/6830657154/extensionFeedItems/163933349386"
   sitelink_feed_item {
     link_text: "Site - Sukienki"
     line1: "Najlepsze sukienki online"
     line2: "Zobacz na Site"
     final_urls: "https://domain.com/pl/"
   }
   status: REMOVED
   extension_type: SITELINK
   id: 163933349386
 },
 extension_feed_item {
   resource_name: "customers/6830657154/extensionFeedItems/163933349389"
   sitelink_feed_item {
     link_text: "Site - Spodnie"
     line1: "Najlepsze sukienki online"
     line2: "Zobacz na Site"
     final_urls: "https://domain.com/pl/"
   }
   status: REMOVED
   extension_type: SITELINK
   id: 163933349389
 },
 extension_feed_item {
   resource_name: "customers/6830657154/extensionFeedItems/164558213554"
   sitelink_feed_item {
     link_text: "Kozaki damskie"
     final_urls: "https://domain.com/pl/kozaki-damskie/2a/"
   }
   status: REMOVED
   extension_type: SITELINK
   id: 164558213554
   targeted_ad_group: "customers/6830657154/adGroups/118059809733"
 },
 extension_feed_item {
   resource_name: "customers/6830657154/extensionFeedItems/164580651119"
   sitelink_feed_item {
     link_text: "Buty zimowe"
     final_urls: "https://domain.com/pl/buty-zimowe-damskie/2a/"
   }
   status: REMOVED
   extension_type: SITELINK
   id: 164580651119
   targeted_ad_group: "customers/6830657154/adGroups/118059809653"
 },
 extension_feed_item {
   resource_name: "customers/6830657154/extensionFeedItems/164580652343"
   sitelink_feed_item {
     link_text: "Buty zimowe"
     final_urls: "https://domain.com/pl/buty-zimowe-damskie/2a__zapiecie-buta-zamek/"
   }
   status: REMOVED
   extension_type: SITELINK
   id: 164580652343
   targeted_ad_group: "customers/6830657154/adGroups/118059809693"
 },
 extension_feed_item {
   resource_name: "customers/6830657154/extensionFeedItems/164580797910"
   sitelink_feed_item {
     link_text: "Kozaki damskie"
     final_urls: "https://domain.com/pl/kozaki-damskie/2a__zapiecie-buta-zamek/"
   }
   status: REMOVED
   extension_type: SITELINK
   id: 164580797910
   targeted_ad_group: "customers/6830657154/adGroups/118059809893"
 },
 extension_feed_item {
   resource_name: "customers/6830657154/extensionFeedItems/164584020051"
   sitelink_feed_item {
     link_text: "Buty zimowe"
     final_urls: "https://domain.com/pl/buty-zimowe-damskie/2a/"
   }
   status: REMOVED
   extension_type: SITELINK
   id: 164584020051
   targeted_ad_group: "customers/6830657154/adGroups/120081592244"
 },
 extension_feed_item {
   resource_name: "customers/6830657154/extensionFeedItems/164584020054"
   sitelink_feed_item {
     link_text: "Buty zimowe"
     final_urls: "https://domain.com/pl/buty-zimowe-damskie/2a__zapiecie-buta-zamek/"
   }
   status: REMOVED
   extension_type: SITELINK
   id: 164584020054
   targeted_ad_group: "customers/6830657154/adGroups/120081592284"
 },
 extension_feed_item {
   resource_name: "customers/6830657154/extensionFeedItems/164584020057"
   sitelink_feed_item {
     link_text: "Kozaki damskie"
     final_urls: "https://domain.com/pl/kozaki-damskie/2a/"
   }
   status: REMOVED
   extension_type: SITELINK
   id: 164584020057
   targeted_ad_group: "customers/6830657154/adGroups/120081592444"
 },
 extension_feed_item {
   resource_name: "customers/6830657154/extensionFeedItems/164584020060"
   sitelink_feed_item {
     link_text: "Kozaki damskie"
     final_urls: "https://domain.com/pl/kozaki-damskie/2a__zapiecie-buta-zamek/"
   }
   status: REMOVED
   extension_type: SITELINK
   id: 164584020060
   targeted_ad_group: "customers/6830657154/adGroups/120081592484"
 },
 extension_feed_item {
   resource_name: "customers/6830657154/extensionFeedItems/165473800714"
   sitelink_feed_item {
     link_text: "Buty zimowe"
     final_urls: "https://domain.com/pl/buty-zimowe-damskie/2a__zapiecie-buta-zamek/"
   }
   status: ENABLED
   extension_type: SITELINK
   id: 165473800714
   targeted_ad_group: "customers/6830657154/adGroups/121584987794"
 },
 extension_feed_item {
   resource_name: "customers/6830657154/extensionFeedItems/165495370142"
   sitelink_feed_item {
     link_text: "Buty zimowe"
     final_urls: "https://domain.com/pl/buty-zimowe-damskie/2a/"
   }
   status: ENABLED
   extension_type: SITELINK
   id: 165495370142
   targeted_ad_group: "customers/6830657154/adGroups/121584987754"
 },
 extension_feed_item {
   resource_name: "customers/6830657154/extensionFeedItems/165495385895"
   sitelink_feed_item {
     link_text: "Kozaki damskie"
     final_urls: "https://domain.com/pl/kozaki-damskie/2a/"
   }
   status: ENABLED
   extension_type: SITELINK
   id: 165495385895
   targeted_ad_group: "customers/6830657154/adGroups/121584987834"
 },
 extension_feed_item {
   resource_name: "customers/6830657154/extensionFeedItems/165495389693"
   sitelink_feed_item {
     link_text: "Kozaki damskie"
     final_urls: "https://domain.com/pl/kozaki-damskie/2a__zapiecie-buta-zamek/"
   }
   status: ENABLED
   extension_type: SITELINK
   id: 165495389693
   targeted_ad_group: "customers/6830657154/adGroups/121584987994"
 }]
BBalazsJurgen commented 3 years ago

Looking over the discussion here it seems as though the main problem is the error:

"Field 'resource_name' cannot be modified by 'CREATE' operation., at mutate_operations[50].extension_feed_item_operation.create.resource_name" and response type "N/A"

Actually, I believe it's kind of both, because: Batch job #0 has a status "Field 'resource_name' cannot be modified by 'CREATE' operation., at mutate_operations[0].extension_feed_item_operation.create.resource_name" and response type "N/A" --> This states that you are not allowed to create an ExtensionFeedItem with a Temporary ID (negative ID in the resource_name). This is quite serious as one would use temporary IDs for a variety of reasons in a batch job, and not being able to use them for an entity is strange.

Also, I've tried this with a couple of other elements (Ad Groups, Keywords, etc.) and they all work just fine with what is basically the same code, using temporary IDs. As far as I can tell, this issue is only present for ExtensionFeedItems.

This however: Batch job #1 has a status "Multiple errors in ‘details’. First error: Field must be set., at mutate_operations[1].ad_group_extension_setting_operation" and response type "N/A" --> This however simply doesn't make sense. I reckon it might have something to do with providing the ExtensionSettingOperation an invalid resource_name (that is, an inexistant extensionfeeditem resource_name, as you provide one with a temporary id that wasn't set in the first operation beforehand due to the first error.

@pySilver I'm going to test and try assigning a .id field to the ExtensionFeedItem, but note that in your response, for example: extension_feed_item { resource_name: "customers/6830657154/extensionFeedItems/165495385895" sitelink_feed_item { link_text: "Kozaki damskie" final_urls: "https://domain.com/pl/kozaki-damskie/2a/" } status: ENABLED extension_type: SITELINK id: 165495385895 targeted_ad_group: "customers/6830657154/adGroups/121584987834" }, "targeted_ad_group" is actually set by the ExtensionFeedItemOperation and not the ExtensionSettingOperation. See here: extension_feed_item.targeted_ad_group = ad_group_resource_name as in your original code. I'm not sure if this ACTUALLY assigns that extension to the particular Ad Group, though there could be miracles I'm not aware of. For in this case, what would be the purpose of ExtensionSettingOperations? Also, extension_feed_item.targeted_ad_group doesn't look like you could pass a list to it, and it should very well be possible to assign an ad extension to multiple ad groups. So this is sort of a mistery field to me, though I did set it in my code the same way you did.

BenRKarl commented 3 years ago

@pySilver Thank you, sorry do you have one that generates the error:

Field 'resource_name' cannot be modified by 'CREATE' operation.

And for the error here:

Field must be set., at mutate_operations[45].ad_group_extension_setting_operation

What did you do differently from the request that returns the first resource_name error?

BenRKarl commented 3 years ago

@BBalazsJurgen You're correct that setting the resource name on an ExtensionFeedItem to use temporary IDs should be possible, it appears this might be a bug, hence why I'm looking for a batch job ID that generates this error so I can query our logs for a root cause.

Also setting the id field shouldn't be possible, I'd expect the API to return an error for that.

BBalazsJurgen commented 3 years ago

@BenRKarl Here's a fresh batch job ID with this error: Created a batch job with resource name "customers/2104968634/batchJobs/4493366812" 2 mutate operations have been added so far. Next sequence token for adding next operations is 71052bed4a20ea4726826373ef415064 Batch job with resource name "customers/2104968634/batchJobs/4493366812" has been executed. Batch job with resource name "customers/2104968634/batchJobs/4493366812" has finished. Now, printing its results... Batch job #0 has a status "Field 'resource_name' cannot be modified by 'CREATE' operation., at mutate_operations[0].extension_feed_item_operation.create.resource_name" and response type "N/A" Batch job #1 has a status "Multiple errors in ‘details’. First error: Field must be set., at mutate_operations[1].ad_group_extension_setting_operation" and response type "N/A"

The BatchJobID is 4493366812.

pySilver commented 3 years ago

@BenRKarl Batch Job ID I've shared uses .id property with TemporaryID in it; It's not supposed to work, yeah but it works. Using .resource_name gives Field 'resource_name' cannot be modified by 'CREATE' operation.

This one isn't clear for me Field must be set., at mutate_operations[45].ad_group_extension_setting_operation at all. I'm setting it as in original topic:

Data:

# feed_item_resource_names:
['customers/<hidden>/extensionFeedItems/-10',
 'customers/<hidden>/extensionFeedItems/-18',
 'customers/<hidden>/extensionFeedItems/-26',
 'customers/<hidden>/extensionFeedItems/-34']

Method:

    def _build_ad_group_extension_settings_operation(
        self, ad_group_resource_name: str, feed_item_resource_names: List[str]
    ) -> AdGroupExtensionSettingOperation:
        """
        Builds Ad Group extension setting operation
        :param ad_group_resource_name: Ad Group resource name
        :return: AdGroupExtensionSettingOperation
        """
        ad_group_ext_setting_operation = self.client.get_type(
            "AdGroupExtensionSettingOperation", version=GOOGLE_ADS_API_VERSION
        )
        extension_type_enum = self.client.get_type("ExtensionTypeEnum", version=GOOGLE_ADS_API_VERSION)

        ad_group_ext_setting = ad_group_ext_setting_operation.create
        ad_group_ext_setting.ad_group = ad_group_resource_name
        ad_group_ext_setting.extension_type = extension_type_enum.SITELINK
        ad_group_ext_setting.extension_feed_items.extend(feed_item_resource_names)
        return ad_group_ext_setting_operation
pySilver commented 3 years ago

@BenRKarl do you have any updates in this matter?

pySilver commented 3 years ago

I'm wondering if thats allowed to call AdGroupExtensionSettingOperation.create in case I want to replace all feed items. Or should I always call .update operation (?)

pySilver commented 3 years ago

I'm not sure its related to this, but I'm getting unexpected error like this from time to time:

Batch job #32000 has a status "The number of entries in the request exceeds the system limit., at mutate_operations[32000].ad_group_extension_setting_operation" and response type "N/A"

while operation is:

ad_group_extension_setting_operation {
  create {
    extension_type: SITELINK
    ad_group: "customers/7159640803/adGroups/123301052590"
    extension_feed_items: "customers/7159640803/extensionFeedItems/167244317982"
    extension_feed_items: "customers/7159640803/extensionFeedItems/167244317985"
    extension_feed_items: "customers/7159640803/extensionFeedItems/167244317988"
    extension_feed_items: "customers/7159640803/extensionFeedItems/167244317991"
    extension_feed_items: "customers/7159640803/extensionFeedItems/167244317994"
    extension_feed_items: "customers/7159640803/extensionFeedItems/167244317997"
    extension_feed_items: "customers/7159640803/extensionFeedItems/167244318000"
  }
}

and current state of this ad group ad extension setting is:

[campaign {
   resource_name: "customers/7159640803/campaigns/12552642226"
   status: ENABLED
   name: "[B] Geox"
   id: 12552642226
 }
 ad_group {
   resource_name: "customers/7159640803/adGroups/123301052590"
   status: ENABLED
   id: 123301052590
   name: "[B] Buty dzieci\304\231ce Geox z syntetyku"
   cpc_bid_micros: 10000
 }
 ad_group_extension_setting {
   resource_name: "customers/7159640803/adGroupExtensionSettings/123301052590~SITELINK"
   extension_type: SITELINK
   ad_group: "customers/7159640803/adGroups/123301052590"
   extension_feed_items: "customers/7159640803/extensionFeedItems/167244317982"
   extension_feed_items: "customers/7159640803/extensionFeedItems/167244317985"
   extension_feed_items: "customers/7159640803/extensionFeedItems/167244317988"
   extension_feed_items: "customers/7159640803/extensionFeedItems/167244317991"
   extension_feed_items: "customers/7159640803/extensionFeedItems/167244317997"
   extension_feed_items: "customers/7159640803/extensionFeedItems/167244318000"
 }]
BBalazsJurgen commented 3 years ago

I think this is unrelated. What seems strange is that you have extension_feed_items set multiple times. I think it is more like an iterable, a list. See here: https://developers.google.com/google-ads/api/reference/rpc/v6/AdGroupExtensionSetting

Maybe try setting the field the way headlines are set of responsive search ads. Check the example file for that.

pySilver commented 3 years ago

@BBalazsJurgen yeah, its iterable – its just how console prints it by default (kinda useful side of Google Ads API client)

pySilver commented 3 years ago

@BBalazsJurgen did you get any response from Google Ads support yet?

BBalazsJurgen commented 3 years ago

All I got was a private response that went like this:

I can confirm that this is an issue at our end. It is related to how Batch jobs handle ExtensionFeedItem. We will work on a fix, but it might take a while. If it is not critical for you to use Batch jobs, you can use GoogleAdsService::Mutate() instead. That service should handle the same set of operations correctly.

That is, I think we might expect an update in v7 of the API whenever that comes out, and until then, no Batch Jobs with temporary IDs for extensions, have to work with Mutates and their responses.

pySilver commented 3 years ago

thanks for headsup!

BenRKarl commented 3 years ago

Hi all - just confirming what @BBalazsJurgen said. Right now it appears to be an oversight that ExtensionFeedItems can't be created using temporary IDs in a batch job. The team is checking whether the change can be made easily, I'm not sure when it might be fixed in the API but I'll try to keep this thread updated.

BenRKarl commented 3 years ago

Hi all - Following up on this issue. It seems that the fix here is not as trivial as I'd hoped, and right now there isn't a specific timeline for a fix. I'm going to close this since it's not specifically related to the Python library here, it affects all other libraries as well.

For now the best workaround is to create the extension feed items using GoogleAdsService.mutate or in batch jobs without using a temporary ID.