equalsraf / win32yank

Windows clipboard tool
ISC License
457 stars 22 forks source link

Do not expect UTF-8 from `stdin` #6

Open kud1ing opened 6 years ago

kud1ing commented 6 years ago

On a German Windows this

echo ö|win32yank –i

gives something like

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error { repr: Custom(Custom { kind: InvalidData, error: StringError("stream did not contain valid UTF-8") }) }', src\libcore\result.rs:859 note: Run with `RUST_BACKTRACE=1` for a backtrace. 

A colleague has this issue when win32yank is called by NeoVim.

equalsraf commented 6 years ago

Indeed, in nvim I think the process output is utf8 (i.e. the win32yank input) - in your case I would assume it is utf16.

Maybe we can use OsString for that read in https://github.com/equalsraf/win32yank/blob/master/src/main.rs#L127

equalsraf commented 5 years ago

It is a bit trickier than that. These are the cases I am aware of

First, the one you mentioned happens because ö uses whichever codepage the console is using. It is possible to decode this by checking the current encoding used by the console

echo ö|win32yank –i

However it this could also fail if the data actually comes from a file. There is nothing we can do in this case for detection - it all depends on the file encoding.

cat test2.txt|win32yank –i

Colleagues have this issue when win32yank is called by NeoVim.

I assume this is possible if the fileencoding is not utf8. Perhaps the easiest way is to force encode as utf8 on the nvim side (assuming this is correct).