search

samccauley / addon-hassiogooglebackup

A hass.io add-on to backup snapshots to Google Drive
MIT License
90 stars 12 forks source link

Redirected but the response is missing a Location: header. #34

Closed mikepruett3 closed 4 years ago

mikepruett3 commented 4 years ago

Getting the following error when calling http://:8055/gb/doBackup

INFO:root:No local_settings to import
WARNING:django.request:Not Found: /
WARNING:django.request:Not Found: /favicon.ico
INFO:oauth2client.client:Refreshing access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.client:Refreshing access_token
WARNING:googleapiclient._helpers:build() takes at most 2 positional arguments (3 given)
INFO:googleapiclient.discovery:URL being requested: GET https://www.googleapis.com/discovery/v1/apis/drive/v3/rest
INFO:googleapiclient.discovery:URL being requested: GET https://www.googleapis.com/drive/v3/files?q=name%3D%276218a928.tar%27+and+%27<redacted>%27+in+parents+and+trashed+%3D+false&spaces=drive&fields=files%28id%2C+name%29&alt=json
INFO:root:Backing up /backup/6218a928.tar to <redacted>
INFO:googleapiclient.discovery:URL being requested: POST https://www.googleapis.com/upload/drive/v3/files?alt=json&uploadType=resumable
ERROR:root:Traceback (most recent call last):
  File "/googlebackup/gb/views.py", line 77, in doBackup
    backupResult = backupFiles(fromPattern, backupDirID, request.build_absolute_uri('/'))
  File "/googlebackup/gbcommon.py", line 263, in backupFiles
    backupFile(file, backupDirID, drive_service, MIMETYPE, TITLE, DESCRIPTION)
  File "/googlebackup/gbcommon.py", line 163, in backupFile
    body=body, media_body=media_body).execute()
  File "/usr/local/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/googleapiclient/http.py", line 813, in execute
    _, body = self.next_chunk(http=http, num_retries=num_retries)
  File "/usr/local/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/googleapiclient/http.py", line 974, in next_chunk
    headers=headers)
  File "/usr/local/lib/python3.7/site-packages/oauth2client/transport.py", line 175, in new_request
    redirections, connection_type)
  File "/usr/local/lib/python3.7/site-packages/oauth2client/transport.py", line 282, in request
    connection_type=connection_type)
  File "/usr/local/lib/python3.7/site-packages/httplib2/__init__.py", line 1994, in request
    cachekey,
  File "/usr/local/lib/python3.7/site-packages/httplib2/__init__.py", line 1690, in _request
    content,
httplib2.RedirectMissingLocation: Redirected but the response is missing a Location: header.

INFO:root:googlebackup result: {'errorMessage': 'Redirected but the response is missing a Location: header.'}
ERROR:django.request:Internal Server Error: /gb/doBackup

Any help would be appreciated!

samccauley commented 4 years ago

I think it may be related to this: https://github.com/googleapis/google-api-python-client/pull/813

It must not effect everyone, because it works okay for me.

Nonetheless, I'll try to patch it today.

samccauley commented 4 years ago

@mikepruett3 I applied a patch (v 1.7.4 of the addon) that I'm hopeful will fix this for you. Please try again, and let me know. Thanks!

mikepruett3 commented 4 years ago

Looks like the same issue after updating to 1.7.4

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] done.
[services.d] starting services
[services.d] done.
GB_DEBUG = false
GB_VERSION = 1.7.4
[2020-06-10 12:53:35 -0500] [200] [INFO] Starting gunicorn 19.9.0
[2020-06-10 12:53:35 -0500] [200] [INFO] Listening at: http://0.0.0.0:8000 (200)
[2020-06-10 12:53:35 -0500] [200] [INFO] Using worker: gevent
[2020-06-10 12:53:35 -0500] [202] [INFO] Booting worker with pid: 202
INFO:root:No local_settings to import
INFO:oauth2client.client:Refreshing access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.client:Refreshing access_token
WARNING:googleapiclient._helpers:build() takes at most 2 positional arguments (3 given)
INFO:googleapiclient.discovery:URL being requested: GET https://www.googleapis.com/discovery/v1/apis/drive/v3/rest
INFO:googleapiclient.discovery:URL being requested: GET https://www.googleapis.com/drive/v3/files?q=name%3D%276218a928.tar%27+and+%271<redacted>%27+in+parents+and+trashed+%3D+false&spaces=drive&fields=files%28id%2C+name%29&alt=json
INFO:root:Backing up /backup/6218a928.tar to <redacted>
INFO:googleapiclient.discovery:URL being requested: POST https://www.googleapis.com/upload/drive/v3/files?alt=json&uploadType=resumable
ERROR:root:Traceback (most recent call last):
  File "/googlebackup/gb/views.py", line 77, in doBackup
    backupResult = backupFiles(fromPattern, backupDirID, request.build_absolute_uri('/'))
  File "/googlebackup/gbcommon.py", line 263, in backupFiles
    backupFile(file, backupDirID, drive_service, MIMETYPE, TITLE, DESCRIPTION)
  File "/googlebackup/gbcommon.py", line 163, in backupFile
    body=body, media_body=media_body).execute()
  File "/usr/local/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/googleapiclient/http.py", line 862, in execute
    _, body = self.next_chunk(http=http, num_retries=num_retries)
  File "/usr/local/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/googleapiclient/http.py", line 1037, in next_chunk
    self.resumable_uri, method="PUT", body=data, headers=headers
  File "/usr/local/lib/python3.7/site-packages/oauth2client/transport.py", line 175, in new_request
    redirections, connection_type)
  File "/usr/local/lib/python3.7/site-packages/oauth2client/transport.py", line 282, in request
    connection_type=connection_type)
  File "/usr/local/lib/python3.7/site-packages/httplib2/__init__.py", line 1994, in request
    cachekey,
  File "/usr/local/lib/python3.7/site-packages/httplib2/__init__.py", line 1690, in _request
    content,
httplib2.RedirectMissingLocation: Redirected but the response is missing a Location: header.
rcont commented 4 years ago

Same error after updating to 1.7.4.

VACIndustries commented 4 years ago

Also seeing the same error on 1.7.4

rcont commented 4 years ago

Some news on this problem?

Ameglialuna commented 4 years ago

1.7.4 and same error

tof92130 commented 4 years ago

Same error with 1.7.4

httplib2.RedirectMissingLocation: Redirected but the response is missing a Location: header. INFO:root:googlebackup result: {'errorMessage': 'Redirected but the response is missing a Location: header.'} DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): hassio:80 DEBUG:urllib3.connectionpool:http://hassio:80 "POST /homeassistant/api/services/mqtt/publish HTTP/1.1" 200 2 DEBUG:root:<Response [200]> Internal Server Error: /gbdoBackup ERROR:django.request:Internal Server Error: /gbdoBackup

flipdream commented 4 years ago

Same error: httplib2.RedirectMissingLocation: Redirected but the response is missing a Location: header.

INFO:root:googlebackup result: {'errorMessage': 'Redirected but the response is missing a Location: header.'} DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): hassio:80 DEBUG:urllib3.connectionpool:http://hassio:80 "POST /homeassistant/api/services/mqtt/publish HTTP/1.1" 200 2 DEBUG:root:<Response [200]> Internal Server Error: /gb/doBackup ERROR:django.request:Internal Server Error: /gb/doBackup

samccauley commented 4 years ago

Sorry guys. I was on vacation the past 11 days. Just got back. I've tried to reproduce this, but cannot. I suspect it has something to do with google's resumeable uploads feature which only kicks in under certain circumstances. I'll try patching to the latest of the google python APIs and related packages.

samccauley commented 4 years ago

@mikepruett3 , @rcont , @VACIndustries , @Ameglialuna , @tof92130 , @flipdream I apologize for the inconvenience. Please try version 1.7.5. Since I'm unable to reproduce the problem, I appreciate your patience with this.

rcont commented 4 years ago

Hi, thanks Scott for the update, but even with version 1.7.5 the backup does not work and the error is the same. I also tried to create a new directory on Google Drive and change the addon configuration, but it doesn't work.

Now I also try to uninstall and install the addon by also deleting the addon permissions on the google account. Then I will let you know

Update. Even reinstalling the addon and deleting the permissions on the Goggle account, unfortunately, does not work!

mikepruett3 commented 4 years ago

Same error, but I did get some additional output as well. Not sure if this is related or not.

...
INFO:root:googlebackup result: {'errorMessage': 'Redirected but the response is missing a Location: header.'}
ERROR:django.request:Internal Server Error: /gb/doBackup
WARNING:django.request:Not Found: /favicon.ico
sunfos2019 commented 4 years ago

Same error on 1.7.5

INFO:root:Backing up /backup/af3d4ab8.tar to 1_1234567890xxxxx_1
ERROR:root:Traceback (most recent call last):
  File "/googlebackup/gb/views.py", line 77, in doBackup
    backupResult = backupFiles(fromPattern, backupDirID, request.build_absolute_uri('/'))
  File "/googlebackup/gbcommon.py", line 263, in backupFiles
    backupFile(file, backupDirID, drive_service, MIMETYPE, TITLE, DESCRIPTION)
  File "/googlebackup/gbcommon.py", line 163, in backupFile
    body=body, media_body=media_body).execute()
  File "/usr/local/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/googleapiclient/http.py", line 871, in execute
    _, body = self.next_chunk(http=http, num_retries=num_retries)
  File "/usr/local/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/googleapiclient/http.py", line 1046, in next_chunk
    self.resumable_uri, method="PUT", body=data, headers=headers
  File "/usr/local/lib/python3.7/site-packages/oauth2client/transport.py", line 175, in new_request
    redirections, connection_type)
  File "/usr/local/lib/python3.7/site-packages/oauth2client/transport.py", line 282, in request
    connection_type=connection_type)
  File "/usr/local/lib/python3.7/site-packages/httplib2/__init__.py", line 1994, in request
    cachekey,
  File "/usr/local/lib/python3.7/site-packages/httplib2/__init__.py", line 1690, in _request
    content,
httplib2.RedirectMissingLocation: Redirected but the response is missing a Location: header.
INFO:root:googlebackup result: {'errorMessage': 'Redirected but the response is missing a Location: header.'}
ERROR:django.request:Internal Server Error: /gb/doBackup
WARNING:django.request:Not Found: /favicon.ico
Ameglialuna commented 4 years ago

Yes, 1.7.5. and same error:

httplib2.RedirectMissingLocation: Redirected but the response is missing a Location: header. INFO:root:googlebackup result: {'errorMessage': 'Redirected but the response is missing a Location: header.'} ERROR:django.request:Internal Server Error: /gb/doBackup

tof92130 commented 4 years ago

Problem remains, here is my log

GB_SLUG = 3833edd4_googlebackup [2020-06-24 13:40:24 +0200] [203] [INFO] Starting gunicorn 19.9.0 [2020-06-24 13:40:24 +0200] [203] [INFO] Listening at: http://0.0.0.0:8000 (203) [2020-06-24 13:40:24 +0200] [203] [INFO] Using worker: gevent [2020-06-24 13:40:24 +0200] [205] [INFO] Booting worker with pid: 205 INFO:root:No local_settings to import DEBUG:root:backup fromPattern: /backup/.tar DEBUG:root:backup backupDirID: 1Np5Cfy6SrCyalgYH0XCjGjBmGVm4ctDC DEBUG:root:backup user_agent: http://hass.peyret.fr:8055/ INFO:oauth2client.client:Refreshing access_token INFO:oauth2client.transport:Attempting refresh to obtain initial access_token INFO:oauth2client.client:Refreshing access_token WARNING:googleapiclient._helpers:build() takes at most 2 positional arguments (3 given) DEBUG:googleapiclient.discovery:URL being requested: GET https://www.googleapis.com/discovery/v1/apis/drive/v3/rest DEBUG:googleapiclient.discovery:URL being requested: GET https://www.googleapis.com/drive/v3/files?q=name%3D%270cccd47a.tar%27+and+%271Np5Cfy6SrCyalgYH0XCjGjBmGVm4ctDC%27+in+parents+and+trashed+%3D+false&spaces=drive&fields=files%28id%2C+name%29&alt=json INFO:root:Backing up /backup/0cccd47a.tar to 1Np5Cfy6SrCyalgYH0XCjGjBmGVm4ctDC DEBUG:root:drive_service = <googleapiclient.discovery.Resource object at 0xb4ba65d0> DEBUG:root:MIMETYPE = application/tar DEBUG:root:TITLE = Hassio Snapshot DEBUG:root:DESCRIPTION = Hassio Snapshot backup copy DEBUG:root:media_body: <googleapiclient.http.MediaFileUpload object at 0xb4d03810> DEBUG:googleapiclient.discovery:URL being requested: POST https://www.googleapis.com/upload/drive/v3/files?alt=json&uploadType=resumable ERROR:root:Traceback (most recent call last): File "/googlebackup/gb/views.py", line 77, in doBackup backupResult = backupFiles(fromPattern, backupDirID, request.build_absolute_uri('/')) File "/googlebackup/gbcommon.py", line 263, in backupFiles backupFile(file, backupDirID, drive_service, MIMETYPE, TITLE, DESCRIPTION) File "/googlebackup/gbcommon.py", line 163, in backupFile body=body, media_body=media_body).execute() File "/usr/local/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper return wrapped(args, *kwargs) File "/usr/local/lib/python3.7/site-packages/googleapiclient/http.py", line 871, in execute _, body = self.next_chunk(http=http, num_retries=num_retries) File "/usr/local/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper return wrapped(args, **kwargs) File "/usr/local/lib/python3.7/site-packages/googleapiclient/http.py", line 1046, in next_chunk self.resumable_uri, method="PUT", body=data, headers=headers File "/usr/local/lib/python3.7/site-packages/oauth2client/transport.py", line 175, in new_request redirections, connection_type) File "/usr/local/lib/python3.7/site-packages/oauth2client/transport.py", line 282, in request connection_type=connection_type) File "/usr/local/lib/python3.7/site-packages/httplib2/init.py", line 1994, in request cachekey, File "/usr/local/lib/python3.7/site-packages/httplib2/init.py", line 1690, in _request content, httplib2.RedirectMissingLocation: Redirected but the response is missing a Location: header. INFO:root:googlebackup result: {'errorMessage': 'Redirected but the response is missing a Location: header.'} DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): hassio:80 DEBUG:urllib3.connectionpool:http://hassio:80 "POST /homeassistant/api/services/mqtt/publish HTTP/1.1" 200 2 DEBUG:root:<Response [200]> Internal Server Error: /gbdoBackup ERROR:django.request:Internal Server Error: /gbdoBackup Not Found: /favicon.ico WARNING:django.request:Not Found: /favicon.ico

samccauley commented 4 years ago

@mikepruett3 , @rcont , @VACIndustries , @Ameglialuna , @tof92130 , @flipdream

TLDR: try 1.7.6 please.

Okay. I rolled back the code that introduced the problem in the first place (it was an update to httplib2 applied to address a vulnerability). When I did that update, it exposed a compatibility issue with google's API and the newer version of httplib2. I had hoped that updating google API would fix it because I hated the idea of rolling back httplib2. But, I relented so I could get this working for you guys. Version 1.7.6 is available now with the older version of httplib2 in it. Please give it a shot. Thank you.

tof92130 commented 4 years ago

It works with 1.7.6

sunfos2019 commented 4 years ago

Works with 1.7.6

rcont commented 4 years ago

I confirm, that with version 1.7.6 everything is back to working properly. I'm sorry you had to go back to the old version of httplib2 and thank you for doing it to make the addon work for us. Thanks a lot Scott

Roberto

Ameglialuna commented 4 years ago

It works with 1.7.6, thank you!