SSLEOFError - gyb quits with unhandled exception after processing 30k out of 50k emails

adam-ah commented 2 years ago

Please confirm the following:

I have upgraded to the latest GYB release from https://github.com/jay0lee/got-your-back/releases and I still have this issue. Yes.

Got Your Back 1.55
https://git.io/gyb
Jay Lee - jay0lee@gmail.com
Python 3.10.2 64-bit final
google-api-client 2.36.0
Linux-5.10.60.1-microsoft-standard-WSL2-x86_64-with-glibc2.31 x86_64
Path: /home/adam/bin/gyb
ConfigPath: /home/adam/bin/gyb
OpenSSL 3.0.1 14 Dec 2021
gmail.googleapis.com connects using TLSv1.3 TLS_AES_256_GCM_SHA384

I am typing the command as described in the GAM Wiki at https://github.com/jay0lee/got-your-back/wiki Yes.

Full steps to reproduce the issue: I don't have a specific easy-to-reproduce and isolated bug. Hopefully a more robust exception handling / retry mechanism around the problem below would solve it.

Steps I took:

Create a full backup of a Google custom domain account locally (completes without problems).
Start restoring the backup to another @gmail address
After a few hours and processing ~30k emails, gyb experienced an unhandled exception and quit as below

gyb --email XXX@gmail.com --action restore --local-folder GYB-GMail-Backup-X@X.com

[...]

Select the actions you wish GYB to be able to perform for XXX@gmail.com

[ ]  0)  Gmail Backup And Restore - read/write mailbox access
[ ]  1)  Gmail Backup Only - read-only mailbox access
[ ]  2)  Gmail Restore Only - write-only mailbox access and label management
[*]  3)  Gmail Full Access - read/write mailbox access and message purge
[ ]  4)  No Gmail Access

[*]  5)  Groups Restore - write to G Suite Groups Archive
[*]  6)  Storage Quota - Drive app config scope used for --action quota

      7)  Continue
7

Go to the following link in your browser:

        https://gyb-shortn.jaylee.us/XXX

Enter verification code: XXX

Using backup folder GYB-GMail-Backup-X@X.com
restoring 1.62mb message (20221/51615)
ERROR: 400: Invalid attachment. Please check https://support.google.com/mail/answer/6590.. Skipping message restore.
restoring 15 messages (25827/51615)
ERROR: 400: Invalid attachment. Please check https://support.google.com/mail/answer/6590.. Skipping message restore.
restoring 15 messages (27460/51615)
ERROR: 400: Invalid attachment. Please check https://support.google.com/mail/answer/6590.. Skipping message restore.
restoring 15 messages (28936/51615)
ERROR: 400: Invalid attachment. Please check https://support.google.com/mail/answer/6590.. Skipping message restore.
restoring 1.37mb message (30608/51615)
ERROR: 400: Invalid attachment. Please check https://support.google.com/mail/answer/6590.. Skipping message restore.
restoring 15 messages (30620/51615)
ERROR: 400: Invalid attachment. Please check https://support.google.com/mail/answer/6590.. Skipping message restore.
Traceback (most recent call last):)
  File "gyb.py", line 2532, in <module>
  File "gyb.py", line 2072, in main
  File "gyb.py", line 733, in callGAPI
  File "googleapiclient/_helpers.py", line 131, in positional_wrapper
  File "googleapiclient/http.py", line 1565, in execute
  File "googleapiclient/http.py", line 1495, in _execute
  File "gyb.py", line 107, in wrapped_request_method
  File "google_auth_httplib2.py", line 218, in request
  File "httplib2/__init__.py", line 1725, in request
  File "httplib2/__init__.py", line 1441, in _request
  File "httplib2/__init__.py", line 1364, in _conn_request
  File "http/client.py", line 1282, in request
  File "http/client.py", line 1328, in _send_request
  File "http/client.py", line 1277, in endheaders
  File "http/client.py", line 1076, in _send_output
  File "http/client.py", line 998, in send
  File "ssl.py", line 1236, in sendall
  File "ssl.py", line 1205, in send
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:2384)
[15408] Failed to execute script 'gyb' due to unhandled exception!

Expected outcome (what are you trying to do?): As above - restoring local backup to a new address

Actual outcome (what errors or bad behavior do you see instead?): As above - unhandled exception, gyb quit.

jay0lee commented 2 years ago

SSL errors always boil down to a connectivity or firewall issue on your machine or network. Make sure you have reliable connectivity and that no firewall on your machine or network is blocking GYB then try again.

adam-ah commented 2 years ago

@jay0lee, uploading messages may take several hours to days and network errors can and do occur. In fact, network errors are the norm, not the exception when performing long operations.

It is not unreasonable for gyb, similarly to most apps performing long network operations, to implement automatic retry if the error did not occur on the very first attempt (i.e., when the error occurs after a successful start).

I cannot think of any apps that has the ability to resume (just like gyb resumes when manually restarted), but crash rather than implement automatic retries on long network operations - can you?

UnknownAlienTechnologies commented 2 years ago

Can this please be addressed? We are archiving a large number of mailboxes and they cannot complete them.

jay0lee commented 2 years ago

PRs are welcome.

SamuelStevenson commented 2 years ago

I just posted to the Google Group about this same issue. I really hope this can be addressed somehow. Don't have any programming skills, but happy to donate!

mrshu commented 2 years ago

It looks like there is some retry mechanism in the function in question (https://github.com/GAM-team/got-your-back/blob/main/gyb.py#L754) -- I guess that might help a bit and this issue might no longer be relevant?

GAM-team / got-your-back

SSLEOFError - gyb quits with unhandled exception after processing 30k out of 50k emails #356