Google storage push is broken

[tschoonj]

Describe the bug

Trying to upload an image to a google storage bucket does not work with httplib2 >= 0.16.0 due to changes introduced in that version that are currently not handled by the google-api-client-api library.

What currently happens is a 'hanging' upload due to an exception being thrown:

Traceback (most recent call last): 0/168 MB - 00:00:00
  File "/dls_sw/apps/python/anaconda/4.6.14/64/envs/python3.8/bin/sregistry", line 11, in <module>
    load_entry_point('sregistry', 'console_scripts', 'sregistry')()
  File "/home/awf63395/github/sregistry-cli/sregistry/client/__init__.py", line 391, in main
    main(args=args, parser=parser, extra=extra)
  File "/home/awf63395/github/sregistry-cli/sregistry/client/push.py", line 33, in main
    cli.push(path=image, name=args.name, tag=args.tag)
  File "/home/awf63395/github/sregistry-cli/sregistry/main/google_storage/push.py", line 44, in push
    manifest = self._upload(
  File "/home/awf63395/github/sregistry-cli/sregistry/main/google_storage/push.py", line 87, in upload
    progress, response = request.next_chunk()
  File "/dls_sw/apps/python/anaconda/4.6.14/64/envs/python3.8/lib/python3.8/site-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/dls_sw/apps/python/anaconda/4.6.14/64/envs/python3.8/lib/python3.8/site-packages/googleapiclient/http.py", line 986, in next_chunk
    resp, content = http.request(self.resumable_uri, method='PUT',
  File "/dls_sw/apps/python/anaconda/4.6.14/64/envs/python3.8/lib/python3.8/site-packages/oauth2client/transport.py", line 173, in new_request
    resp, content = request(orig_request_method, uri, method, body,
  File "/dls_sw/apps/python/anaconda/4.6.14/64/envs/python3.8/lib/python3.8/site-packages/oauth2client/transport.py", line 280, in request
    return http_callable(uri, method=method, body=body, headers=headers,
  File "/dls_sw/apps/python/anaconda/4.6.14/64/envs/python3.8/lib/python3.8/site-packages/httplib2/__init__.py", line 1982, in request
    (response, content) = self._request(
  File "/dls_sw/apps/python/anaconda/4.6.14/64/envs/python3.8/lib/python3.8/site-packages/httplib2/__init__.py", line 1685, in _request
    raise RedirectMissingLocation(
httplib2.RedirectMissingLocation: Redirected but the response is missing a Location: header.

which is caught by the retry call, so had to remove the decorator to actually get this far :smile:

To Reproduce Steps to reproduce the behavior: install sregistry with google-storage backend support using the latest versions of all required dependencies.

This may also affect other (google-) backends but I haven't tested this.

Version of Singularity and Singularity Registry Client sregistry-client: 0.2.33

Anything else?

Should we pin the version of httplib2 in the conda repo to <= 0.15.0 until the google-api-client library contains a fix?

[vsoch]

Yes pinning the version to what previously worked is a good bet for now. I’d be happy to review a PR for this.

[vsoch]

Fixed with #253

[tschoonj]

This is fixed upstream now, may be useful to change pinning here, and revert the corresponding PR.

[vsoch]

@tschoonj that's a good idea - revert would remove all the commits done for that PR (which I don't think we want to do) but if you have bandwidth in the next few weeks, do you want to open a simple PR to bump the version and change pinning? We could then update the conda force recipe to match.

[tschoonj]

First thing tomorrow morning!

[vsoch]

Woot! Thanks @tschoonj !