Move away from UTF-8 when decoding terminal output?

FredHutch / motuz

Motuz - A web based infrastructure for large scale data movements between on-premise and cloud

MIT License

103 stars 12 forks source link

Move away from UTF-8 when decoding terminal output? #139

Open dirkpetersen opened 5 years ago

dirkpetersen commented 5 years ago

This error message is somehow familiar from Python3 but this error should be coming from rclone ? But if the file name was returned from rclone to celery it may have such an error message

aicioara commented 5 years ago

Oh, that is because I decode everything as UTF-8. Do you use non-UTF-8 characters a lot (at all) in your folder namings?

aicioara commented 5 years ago

Could you give me an example of a folder name that contains non-utf-8 characters?

dirkpetersen commented 5 years ago

I have to ask my colleague mike, I just tried it with german umlauts (Austria) and it worked well

dirkpetersen commented 5 years ago

Mike says: I’m working identifying the specific characters, but these errors seemed to occur when I was browsing directories with leftover Office temporary files having filenames containing ‘$’ and ‘~’.

aicioara commented 5 years ago

@dirkpetersen, is there additional information with regards to how to reproduce this issue?

aicioara commented 5 years ago

This will be discussed tomorrow, moving it to V2