joeyates / imap-backup

Backup and Migrate IMAP Email Accounts
MIT License
1.33k stars 74 forks source link

Migrate is failing #186

Closed TriPSs closed 7 months ago

TriPSs commented 7 months ago

Getting the same error as in #115

/opt/homebrew/Cellar/ruby/3.2.2_1/lib/ruby/gems/3.2.0/gems/net-imap-0.3.4/lib/net/imap.rb:2170:in `get_tagged_response': Character not allowed in mailbox name: '.' (0.001 + 0.170 + 0.169 secs). (Net::IMAP::NoResponseError)
    from /opt/homebrew/Cellar/ruby/3.2.2_1/lib/ruby/gems/3.2.0/gems/net-imap-0.3.4/lib/net/imap.rb:2224:in `block in send_command'
    from /opt/homebrew/Cellar/ruby/3.2.2_1/lib/ruby/3.2.0/monitor.rb:202:in `synchronize'
    from /opt/homebrew/Cellar/ruby/3.2.2_1/lib/ruby/3.2.0/monitor.rb:202:in `mon_synchronize'
    from /opt/homebrew/Cellar/ruby/3.2.2_1/lib/ruby/gems/3.2.0/gems/net-imap-0.3.4/lib/net/imap.rb:2206:in `send_command'
    from /opt/homebrew/Cellar/ruby/3.2.2_1/lib/ruby/gems/3.2.0/gems/net-imap-0.3.4/lib/net/imap.rb:1130:in `create'
    from /opt/homebrew/Cellar/ruby/3.2.2_1/lib/ruby/3.2.0/forwardable.rb:240:in `create'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/client/automatic_login_wrapper.rb:23:in `method_missing'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/account/folder.rb:55:in `block in create'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/retry_on_error.rb:9:in `retry_on_error'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/account/folder.rb:54:in `create'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/migrator.rb:19:in `run'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/cli/transfer.rb:54:in `block in run'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/cli/folder_enumerator.rb:42:in `block in each'
    from <internal:dir>:220:in `glob'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/cli/folder_enumerator.rb:38:in `glob'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/cli/folder_enumerator.rb:38:in `each'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/cli/transfer.rb:51:in `run'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/cli.rb:146:in `migrate'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/thor-1.3.0/lib/thor/command.rb:28:in `run'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/thor-1.3.0/lib/thor/invocation.rb:127:in `invoke_command'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/thor-1.3.0/lib/thor.rb:527:in `dispatch'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/thor-1.3.0/lib/thor/base.rb:584:in `start'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/cli.rb:64:in `start'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/bin/imap-backup:13:in `block in <top (required)>'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/logger.rb:42:in `sanitize_stderr'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/bin/imap-backup:12:in `<top (required)>'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/bin/imap-backup:25:in `load'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/bin/imap-backup:25:in `<main>'

What can I set/do to fix this?

joeyates commented 7 months ago

Hi @TriPSs

Thanks for opening this issue.

Please run the restore with the --verbose parameter and send me the sanitized output.

In the meantime, I'll have a look at the code.

TriPSs commented 7 months ago

Here:

/opt/homebrew/Cellar/ruby/3.2.2_1/lib/ruby/gems/3.2.0/gems/net-imap-0.3.4/lib/net/imap.rb:2170:in `get_tagged_response': Character not allowed in mailbox name: '.' (0.001 + 0.155 + 0.154 secs). (Net::IMAP::NoResponseError)
    from /opt/homebrew/Cellar/ruby/3.2.2_1/lib/ruby/gems/3.2.0/gems/net-imap-0.3.4/lib/net/imap.rb:2224:in `block in send_command'
    from /opt/homebrew/Cellar/ruby/3.2.2_1/lib/ruby/3.2.0/monitor.rb:202:in `synchronize'
    from /opt/homebrew/Cellar/ruby/3.2.2_1/lib/ruby/3.2.0/monitor.rb:202:in `mon_synchronize'
    from /opt/homebrew/Cellar/ruby/3.2.2_1/lib/ruby/gems/3.2.0/gems/net-imap-0.3.4/lib/net/imap.rb:2206:in `send_command'
    from /opt/homebrew/Cellar/ruby/3.2.2_1/lib/ruby/gems/3.2.0/gems/net-imap-0.3.4/lib/net/imap.rb:1130:in `create'
    from /opt/homebrew/Cellar/ruby/3.2.2_1/lib/ruby/3.2.0/forwardable.rb:240:in `create'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/client/automatic_login_wrapper.rb:23:in `method_missing'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/account/folder.rb:55:in `block in create'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/retry_on_error.rb:9:in `retry_on_error'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/account/folder.rb:54:in `create'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/migrator.rb:19:in `run'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/cli/transfer.rb:54:in `block in run'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/cli/folder_enumerator.rb:42:in `block in each'
    from <internal:dir>:220:in `glob'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/cli/folder_enumerator.rb:38:in `glob'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/cli/folder_enumerator.rb:38:in `each'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/cli/transfer.rb:51:in `run'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/cli.rb:146:in `migrate'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/thor-1.3.0/lib/thor/command.rb:28:in `run'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/thor-1.3.0/lib/thor/invocation.rb:127:in `invoke_command'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/thor-1.3.0/lib/thor.rb:527:in `dispatch'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/thor-1.3.0/lib/thor/base.rb:584:in `start'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/cli.rb:64:in `start'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/bin/imap-backup:13:in `block in <top (required)>'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/logger.rb:42:in `sanitize_stderr'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/bin/imap-backup:12:in `<top (required)>'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/bin/imap-backup:25:in `load'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/bin/imap-backup:25:in `<main>'
joeyates commented 7 months ago

Hi @TriPSs

Sorry, please use --verbose twice - it's now necessary to do that to get full client-server debugging

joeyates commented 7 months ago

...also, are you restoring or migrating?

Please include the command you are giving as well as the debug output.

TriPSs commented 7 months ago

Okay, full command here

imap-backup migrate old-email new-email --verbose --verbose
D, [2023-11-20T15:35:27.565887 #32031] DEBUG -- : Creating IMAP instance: mail.privateemail.com, options: {:port=>993, :ssl=>{:ssl_version=>:TLSv1_2}}
C: RUBY0003 CREATE "xxxx.01 of 07-02-24 xxx xxxx"
S: RUBY0003 NO [CANNOT] Character not allowed in mailbox name: '.' (0.001 + 0.150 + 0.149 secs).
/opt/homebrew/Cellar/ruby/3.2.2_1/lib/ruby/gems/3.2.0/gems/net-imap-0.3.4/lib/net/imap.rb:2170:in `get_tagged_response': Character not allowed in mailbox name: '.' (0.001 + 0.150 + 0.149 secs). (Net::IMAP::NoResponseError)
    from /opt/homebrew/Cellar/ruby/3.2.2_1/lib/ruby/gems/3.2.0/gems/net-imap-0.3.4/lib/net/imap.rb:2224:in `block in send_command'
    from /opt/homebrew/Cellar/ruby/3.2.2_1/lib/ruby/3.2.0/monitor.rb:202:in `synchronize'
    from /opt/homebrew/Cellar/ruby/3.2.2_1/lib/ruby/3.2.0/monitor.rb:202:in `mon_synchronize'
    from /opt/homebrew/Cellar/ruby/3.2.2_1/lib/ruby/gems/3.2.0/gems/net-imap-0.3.4/lib/net/imap.rb:2206:in `send_command'
    from /opt/homebrew/Cellar/ruby/3.2.2_1/lib/ruby/gems/3.2.0/gems/net-imap-0.3.4/lib/net/imap.rb:1130:in `create'
    from /opt/homebrew/Cellar/ruby/3.2.2_1/lib/ruby/3.2.0/forwardable.rb:240:in `create'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/client/automatic_login_wrapper.rb:23:in `method_missing'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/account/folder.rb:55:in `block in create'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/retry_on_error.rb:9:in `retry_on_error'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/account/folder.rb:54:in `create'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/migrator.rb:19:in `run'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/cli/transfer.rb:54:in `block in run'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/cli/folder_enumerator.rb:42:in `block in each'
    from <internal:dir>:220:in `glob'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/cli/folder_enumerator.rb:38:in `glob'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/cli/folder_enumerator.rb:38:in `each'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/cli/transfer.rb:51:in `run'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/cli.rb:146:in `migrate'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/thor-1.3.0/lib/thor/command.rb:28:in `run'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/thor-1.3.0/lib/thor/invocation.rb:127:in `invoke_command'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/thor-1.3.0/lib/thor.rb:527:in `dispatch'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/thor-1.3.0/lib/thor/base.rb:584:in `start'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/cli.rb:64:in `start'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/bin/imap-backup:13:in `block in <top (required)>'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/lib/imap/backup/logger.rb:42:in `sanitize_stderr'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/gems/imap-backup-14.3.0/bin/imap-backup:12:in `<top (required)>'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/bin/imap-backup:25:in `load'
    from /opt/homebrew/Cellar/imap-backup/14.3.0/libexec/bin/imap-backup:25:in `<main>'
joeyates commented 7 months ago

OK, you need to see what the delimiter is on the new server:

Here's an example for GMail:

imap-backup remote namespaces YOUR_EMAIL -c YOUR_CONFIG
Name       Prefix     Delimiter
personal   ""         "/"
other      (Not defined)
shared     (Not defined)

When you've got the "personal" delimiter, pass that as the --destination-delimiter parameter:

imap-backup help migrate                                                                                                
Usage:                                                                                                                                                
  imap-backup migrate SOURCE_EMAIL DESTINATION_EMAIL [OPTIONS]                                                                                        

Options:                             
  -c, [--config=CONFIG]                                      # supply the configuration file path (default: ~/.imap-backup/config.json)
  -q, [--quiet], [--no-quiet]                                # silence all output                                                                     
  -v, [--verbose], [--no-verbose]                            # increase the amount of logging. Without this option, the program gives minimal output. 
Using this option once gives more detailed output. Whereas, using this option twice also shows all IMAP network calls
      [--automatic-namespaces], [--no-automatic-namespaces]  # automatically choose delimiters and prefixes
      [--destination-delimiter=DESTINATION-DELIMITER]        # the delimiter for destination folder names
  -d, [--destination-prefix=DESTINATION-PREFIX]              # the prefix (namespace) to add to destination folder names
  -r, [--reset], [--no-reset]                                # DANGER! This option deletes all messages from destination folders before uploading
      [--source-delimiter=SOURCE-DELIMITER]                  # the delimiter for source folder names
  -s, [--source-prefix=SOURCE-PREFIX]                        # the prefix (namespace) to strip from source folder names

Description:                         
  All emails which have been backed up for the "source account" (SOURCE_EMAIL) are                                                                    
  uploaded to the "destination account" (DESTINATION_EMAIL).                                                                                          

  Some configuration may be necessary, as follows:                                                                                                    

  Some IMAP servers use namespaces (i.e. prefixes like "INBOX"), while others, while                                                                  
  others concatenate the names of subfolders with a charater ("delimiter") other than                                                                 
  "/".                               

  In these cases there are two choices.                                    

  You can use the `--automatic-namespaces` option. This wil query the source and                                                                      
  detination servers for their namespace configuration and will adapt paths accordingly.                                                              
  This option requires that both the source and destination servers are available and                                                                 
  work with the provided parameters and authentication.                                                                                               

  If automatic configuration does not work as desired, there are the `--source-prefix=`,                                                              
  `--source-delimiter=`, `--destination-prefix=` and `--destination-delimiter=`                                                                       
  parameters. To check what values you should use, check the output of the `imap-backup                                                               
  remote namespaces EMAIL` command. 
TriPSs commented 7 months ago

Okay, i ran the following:

imap-backup remote namespaces email
Name       Prefix     Delimiter
personal   ""         "/"
other      (Not defined)
shared     (Not defined)

After that I ran imap-backup migrate old_email new-email --destination-delimiter="/" but the error stays the same

joeyates commented 7 months ago

OK, so the problem is not delimiters. You have an actual '.' in a folder name.

That was allowed by the old email server, but not by the new one. So, you'll need to rename the folder name(s) locally and re-run the migration

TriPSs commented 7 months ago

Thanks, renaming the files with a dot in it fixes it!