Doing a batched delete of contacts always returns "If-Match or If-None-Match header or entry etag attribute required" error

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Prepare a ContactsFeed with contacts to be deleted

    request_feed = gdata.contacts.data.ContactsFeed()
    request_feed.AddDelete(entry=contact, batch_id_string='delete')

2. Perform ExecuteBatch

    response_feed = self.gd_client.ExecuteBatch(
            request_feed,
            'https://www.google.com/m8/feeds/contacts/default/full/batch'
    )

What is the expected output? What do you see instead?

Expected output would be for the contacts to be deleted. Instead a 403 response 
with message "If-Match or If-None-Match header or entry etag attribute 
required" is returned.

Even with the recommended workaround of adding the "If-Match: *" header doesn't 
work:

    custom_headers = atom.client.CustomHeaders(**{'If-Match': '*'})
    request_feed = gdata.contacts.data.ContactsFeed()
    request_feed.AddDelete(entry=contact, batch_id_string='delete')
    response_feed = self.gd_client.ExecuteBatch(
            request_feed,
            'https://www.google.com/m8/feeds/contacts/default/full/batch'
    )

What version of the product are you using?

gdata 2.0.18

Please provide any additional information below.

Not sure if it's related, but this started happening when Google required the 
"Contacts API" entry in the Cloud Console UI to be enabled. Before, there was 
no entry for "Contacts API" in the Cloud Console UI, so I guess it was enabled 
by default. But after being added, and it was set to Disabled by default, 
things started breaking.

Other discussions on StackOverflow:
http://stackoverflow.com/questions/23757499/if-match-or-if-none-match-header-or-
entry-etag-attribute-required-error-while-up
http://stackoverflow.com/questions/2989257/if-match-or-if-none-match-header-or-e
ntry-etag-attribute-required-error-when-t?rq=1
http://stackoverflow.com/questions/23576729/getting-if-match-or-if-none-match-he
ader-or-entry-etag-attribute-required-erro

Original issue reported on code.google.com by joh...@gmail.com on 28 May 2014 at 8:19

[GoogleCodeExporter]

Using the `Batch` method with `force=True` is the same result.

    response_feed = self.gd_client.Batch(
        request_feed,
        uri='https://www.google.com/m8/feeds/contacts/default/full/batch',
        force=True
    )

Original comment by joh...@gmail.com on 29 May 2014 at 1:59

[GoogleCodeExporter]

There is a typo on the first code snippet (missing custom_headers param). 
Should read:

    custom_headers = atom.client.CustomHeaders(**{'If-Match': '*'})
    request_feed = gdata.contacts.data.ContactsFeed()
    request_feed.AddDelete(entry=contact, batch_id_string='delete')
    response_feed = self.gd_client.ExecuteBatch(
            request_feed,
            'https://www.google.com/m8/feeds/contacts/default/full/batch',
            custom_headers=custom_headers
    )

Original comment by joh...@gmail.com on 3 Jun 2014 at 3:20

[GoogleCodeExporter]

 im having the same issue

Original comment by administ...@eforcers.com.co on 21 Jun 2014 at 6:40

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

same - having the same issue. there is a solution?

Original comment by tsarfaty...@gmail.com on 28 Jun 2014 at 7:08

[GoogleCodeExporter]

I've been digging at it for a few hours, no progress yet. Here's a sample 
request feed:

<ns0:feed xmlns:ns0="http://www.w3.org/2005/Atom" 
xmlns:ns1="http://schemas.google.com/g/2005" 
xmlns:ns2="http://schemas.google.com/contact/2008" 
xmlns:ns3="http://schemas.google.com/gdata/batch" 
xmlns:ns4="http://www.w3.org/2007/app">
<ns0:entry ns1:etag=""<id>"">
    <ns2:groupMembershipInfo deleted="false" href="http://www.google.com/m8/feeds/groups/<user_email>/base/6"/>
    <ns2:groupMembershipInfo deleted="false" href="http://www.google.com/m8/feeds/groups/<user_email>/base/<id>"/>
    <ns0:id>http://www.google.com/m8/feeds/contacts/<user_email>/base/<id></ns0:id>
    <ns1:name>
        <ns1:fullName>Test Two</ns1:fullName>
        <ns1:familyName>Two</ns1:familyName>
        <ns1:givenName>Test</ns1:givenName>
    </ns1:name>
    <ns0:category scheme="http://schemas.google.com/g/2005#kind" term="http://schemas.google.com/contact/2008#contact"/>
    <ns0:title>Test Two</ns0:title>
    <ns0:updated>2014-06-30T22:13:48.049Z</ns0:updated>
    <ns3:id>delete</ns3:id>
    <ns3:operation type="delete"/>
    <ns0:link href="https://www.google.com/m8/feeds/photos/media/<user_email>/<id>" rel="http://schemas.google.com/contacts/2008/rel#photo" type="image/*"/>
    <ns0:link href="https://www.google.com/m8/feeds/contacts/<user_email>/full/<id>" rel="self" type="application/atom+xml"/>
    <ns0:link href="https://www.google.com/m8/feeds/contacts/<user_email>/full/<id" rel="edit" type="application/atom+xml"/>
    <ns4:edited>2014-06-30T22:13:48.049Z</ns4:edited>
</ns0:entry>
<ns0:entry ns1:etag=""<id>"">
    <ns2:groupMembershipInfo deleted="false" href="http://www.google.com/m8/feeds/groups/<user_email>/base/6"/>
    <ns2:groupMembershipInfo deleted="false" href="http://www.google.com/m8/feeds/groups/<user_email>/base/<id>"/>
    <ns0:id>http://www.google.com/m8/feeds/contacts/<user_email>/base/<id></ns0:id>
    <ns1:name>
        <ns1:fullName>Test One</ns1:fullName>
        <ns1:familyName>One</ns1:familyName>
        <ns1:givenName>Test</ns1:givenName>
    </ns1:name>
    <ns0:category scheme="http://schemas.google.com/g/2005#kind" term="http://schemas.google.com/contact/2008#contact"/>
    <ns0:title>Test One</ns0:title>
    <ns0:updated>2014-06-30T22:13:44.164Z</ns0:updated>
    <ns3:id>delete</ns3:id>
    <ns3:operation type="delete"/>
    <ns0:link href="https://www.google.com/m8/feeds/photos/media/<user_email>/<id>" rel="http://schemas.google.com/contacts/2008/rel#photo" type="image/*"/>
    <ns0:link href="https://www.google.com/m8/feeds/contacts/<user_email>/full/<id>" rel="edit" type="application/atom+xml"/>
    <ns4:edited>2014-06-30T22:13:44.164Z</ns4:edited>
</ns0:entry>
</ns0:feed>

Original comment by joseph.d...@gmail.com on 1 Jul 2014 at 3:03

[GoogleCodeExporter]

I had the same problem, but i found (few days ago), that if you replace in 
request feed the namespace prefix of 
xmlns:ns1="http://schemas.google.com/g/2005" to 'gd' (ns1 > gd), it's working.
You can try it in OAuth 2.0 Playground.

Original comment by jahny.t...@gmail.com on 1 Jul 2014 at 10:47

[GoogleCodeExporter]

Thanks, that worked! I ended up making a modification to gdata/client.py, I 
couldn't figure out a better way to override it from my own code. I'll have to 
do some more testing to make sure it doesn't hurt anything. It should get me by 
for now. Here's the change:

In data/client.py:

 def post(self, entry, uri, auth_token=None, converter=None,
            desired_class=None, **kwargs):
     if converter is None and desired_class is None:
       desired_class = entry.__class__
     http_request = atom.http_core.HttpRequest()
+    entry_string = entry.to_string(get_xml_version(self.api_version))
+    entry_string = entry_string.replace('ns1', 'gd')
     http_request.add_body_part(
-        entry.to_string(get_xml_version(self.api_version)),
+        entry_string,
         'application/atom+xml')
     return self.request(method='POST', uri=uri, auth_token=auth_token,
                         http_request=http_request, converter=converter,
                         desired_class=desired_class, **kwargs)

Original comment by joseph.d...@gmail.com on 2 Jul 2014 at 1:38

[GoogleCodeExporter]

@joseph, Thanks, your patch seems to work for me as well. Will do more tests. 
To keep from modifying gdata/client.py, I did this instead:

    def patched_post(client, entry, uri, auth_token=None, converter=None, desired_class=None, **kwargs):
        if converter is None and desired_class is None:
            desired_class = entry.__class__
        http_request = atom.http_core.HttpRequest()
        entry_string = entry.to_string(gdata.client.get_xml_version(client.api_version))
        entry_string = entry_string.replace('ns1', 'gd')
        http_request.add_body_part(
            entry_string,
            'application/atom+xml')
        return client.request(method='POST', uri=uri, auth_token=auth_token,
                              http_request=http_request, converter=converter,
                              desired_class=desired_class, **kwargs)

    # when it comes time to do a batched delete/update,
    # instead of calling client.ExecuteBatch, I instead directly call patched_post
    patched_post(my_client_instance, request_feed,
        'https://www.google.com/m8/feeds/contacts/default/full/batch')

Original comment by j...@collabspot.com on 16 Jul 2014 at 2:20

[GoogleCodeExporter]

> entry_string.replace('ns1', 'gd')

Is this the correct way to solve the problem?
It will indiscriminately replace all occurrences of 'ns1' from the anywhere in 
the string representation of the entry, including its user-submitted data 
fields.

Original comment by the.matr...@gmail.com on 18 Jul 2014 at 9:40

[GoogleCodeExporter]

Ideally this problem can be fixed in Google's server-side parser code.

<ns0:feed xmlns:ns1="http://schemas.google.com/g/2005" ns1:etag="foobar">

is semantically equivalent to 

<ns0:feed xmlns:gd="http://schemas.google.com/g/2005" gd:etag="foobar">

No?

Original comment by the.matr...@gmail.com on 18 Jul 2014 at 10:02

[GoogleCodeExporter]

Yes, ideally Google fix it server side. But until then, this is the best we 
got. But yes, maybe it should be:

    entry_string.replace('ns1=', 'gd=').replace('ns1:', 'gd:')

Original comment by joh...@gmail.com on 19 Jul 2014 at 12:27

[GoogleCodeExporter]

Each entry in the feed requires an etag for verification of the resource 
version (https://developers.google.com/gdata/docs/2.0/reference). 

The solution is to do a query first to return the etags for your feed, then add 
the respective etags to the update feed.

def make_batch_post_request(spreadsheets_client, feed):
    self = spreadsheets_client
    import atom
    http_request = atom.http_core.HttpRequest()
    http_request.add_body_part(
        feed.to_string(get_xml_version(self.api_version)).replace('update', 'query'),
        'application/atom+xml')
    auth_token = spreadsheets_client.auth_token
    uri = feed.find_edit_link()
    response = self.request('POST', uri=uri, auth_token=auth_token, http_request=http_request, desired_class=feed.__class__)

# <magic>
    for index, entry in enumerate(feed.entry):
        feed.entry[index].etag = response.entry[index].etag
#</magic>

    http_request = atom.http_core.HttpRequest()
    http_request.add_body_part(
        feed.to_string(get_xml_version(self.api_version)),
        'application/atom+xml')
    auth_token = spreadsheets_client.auth_token
    uri = feed.find_edit_link()
    response = self.request('POST', uri=uri, auth_token=auth_token, http_request=http_request, desired_class=feed.__class__)

    return response

Original comment by james.ch...@rightster.com on 29 Aug 2014 at 5:02

[GoogleCodeExporter]

@joseph Thanks, your patch is working :-)

Original comment by Un4getta...@gmail.com on 2 Mar 2015 at 7:57