Error when uploading - UnicodeDecodeError

[SystemZ]

Hello, script seems to be awesome but unfortunately I can't use it because there is some problem with uploading. Direct invoking b2 upload_file works ok. I tried to replace xargs to even simpler for loop for upload step but still no luck, same error.

Log from Proxmox 4.x backup:

INFO: starting new backup job: vzdump 1111 --storage local --compress gzip --remove 0 --node n1 --mode snapshot
INFO: Starting Backup of VM 1111 (qemu)
INFO: status = running
INFO: update VM 1111: -lock backup
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating archive '/var/lib/vz/dump/vzdump-qemu-1111-2016_04_05-18_03_34.vma.gz'
INFO: started backup task '3cea83e9-753d-46ae-b495-66dcee321e55'
INFO: status: 1% (201457664/10737418240), sparse 1% (127156224), duration 3, 67/24 MB/s
INFO: status: 2% (277479424/10737418240), sparse 1% (127238144), duration 6, 25/25 MB/s
INFO: status: 3% (353501184/10737418240), sparse 1% (128651264), duration 9, 25/24 MB/s
INFO: status: 4% (437125120/10737418240), sparse 1% (131043328), duration 12, 27/27 MB/s
INFO: status: 5% (539754496/10737418240), sparse 1% (137904128), duration 15, 34/31 MB/s
INFO: status: 21% (2322464768/10737418240), sparse 17% (1839480832), duration 18, 594/27 MB/s
INFO: status: 33% (3575775232/10737418240), sparse 27% (2984382464), duration 21, 417/36 MB/s
INFO: status: 61% (6629097472/10737418240), sparse 55% (5940805632), duration 24, 1017/32 MB/s
INFO: status: 62% (6712721408/10737418240), sparse 55% (5945184256), duration 27, 27/26 MB/s
INFO: status: 63% (6796345344/10737418240), sparse 55% (5945315328), duration 30, 27/27 MB/s
INFO: status: 100% (10737418240/10737418240), sparse 91% (9850957824), duration 32, 1970/17 MB/s
INFO: transferred 10737 MB in 32 seconds (335 MB/s)
INFO: archive file size: 279MB
INFO: CHECKSUMMING whole tar.
INFO: SPLITTING into chunks sized <=2000000000 byte
INFO: real  0m0.921s
INFO: user  0m0.000s
INFO: sys   0m0.136s
INFO: CHECKSUMMING splits
INFO: Deleting whole file
INFO: ENCRYPTING
INFO: 3.12user 0.16system 0:03.28elapsed 100%CPU (0avgtext+0avgdata 3448maxresident)k
INFO: 32inputs+573200outputs (1major+274minor)pagefaults 0swaps
INFO: Checksumming encrypted splits
INFO: Deleting cleartext splits
INFO: AUTHORIZING AGAINST B2
INFO: Using https://api.backblaze.com
INFO: UPLOADING to B2 with up to 10 parallel uploads.
INFO: Traceback (most recent call last):
INFO:   File "/usr/local/bin/b2", line 1970, in <module>
INFO: Traceback (most recent call last):
INFO:   File "/usr/local/bin/b2", line 1970, in <module>
INFO:     main()
INFO:   File "/usr/local/bin/b2", line 1916, in main
INFO:     main()
INFO:   File "/usr/local/bin/b2", line 1916, in main
INFO:     decoded_argv = decode_sys_argv()
INFO:   File "/usr/local/bin/b2", line 1089, in decode_sys_argv
INFO:     decoded_argv = decode_sys_argv()
INFO:   File "/usr/local/bin/b2", line 1089, in decode_sys_argv
INFO:     return [arg.decode(encoding) for arg in sys.argv]
INFO: UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 5: ordinal not in range(128)
INFO:     return [arg.decode(encoding) for arg in sys.argv]
INFO: UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 5: ordinal not in range(128)
INFO: Something went wrong uploading.
ERROR: Backup of VM 1111 failed - command '/usr/local/bin/vzdump-plugin-b2/vzdump-plugin-upload-b2.sh backup-end snapshot 1111' failed: exit code 10
INFO: Backup job finished with errors
TASK ERROR: job errors

[padelt]

Hi Michał, Just a shot in the dark: is there a non-ascii character in the source or destination path? Philipp

[SystemZ]

Hi Philipp, No idea :) I think not, I'm using default Proxmox dirs and configs. Path looks like this /var/lib/vz/dump file names looks ok too. Maybe when proxmox task is invoking command there are some strange chars. Do you have any idea how to check this effectively? Is there any way to filter/trim this?

[padelt]

Hi Michał, I just added a parameter to the xargs invocation. This should make the whole upload command end up in the log. Maybe we can find something special in there. There should be something like that:

INFO: /usr/local/bin/b2 upload_file ipabackup /rpool/backup/dump/vzdump-qemu-100-2016_04_06-11_13_21.vma.lzo.sha1sums ipapm2/rpool/backup/dump/vzdump-qemu-100-2016_04_06-11_13_21.vma.lzo.sha1sums

I still suspect some non-ascii chars in there. Not that b2 should choke on that, but I had similar problems before. Philipp

[SystemZ]

Thank you, it's very helpful already :)

You were right there is non ascii symbol. Instead of - in bucket name there appears ‑. Strange, I'll try to copy paste this symbol or transfer it without using nano editor.

[SystemZ]

Just mounted sftp on my desktop and edited config with gedit and it works! Thank you very much for fast help :) I'll try to contribute download script for restoring backups.

[padelt]

Great to hear you made it work! Before you do too much work, have a look at https://github.com/padelt/vzdump-plugin-b2/blob/master/vzdump-b2-verify.sh , it leaves the dump file after verifying. After moving it in place, Proxmox VE should pick it up and allow you to restore. As always, testing it advised! ;-)