akiirui / mpv-handler

A protocol handler for mpv. Use mpv and yt-dlp to play video and music from the websites.
MIT License
262 stars 22 forks source link

invalid utf-8 sequence of 1 bytes from index 2 #11

Closed ghost closed 3 years ago

ghost commented 3 years ago

Had mpv-handler from AUR & tamper-monkey on chromium. What's could be the cause & solution ? invalid utf-8 sequence of 1 bytes from index 2

akiirui commented 3 years ago

Please show me these informations:

ghost commented 3 years ago

Please show me these informations:

  • The video URL when this error occurred.
  • The base64 encoded URL (right click on the mpv icon to copy)
  • config.toml & custom.toml (if you created it)

Hey apologize for late reply... Here's the output from your debug build, as per the one you suggested on other opened issue $ ./mpv-handler 'mpv://aHR0cHM6Ly93d3cueW91dHViZS5jb20vd2F0Y2g/dj1NdmJsZGNrQUY5NA==/?downloader=mpv&quality=best' [src/protocol.rs:88] &data_base64 = "aHR0cHM6Ly93d3cueW91dHViZS5jb20vd2F0Y2g/dj1NdmJsZGNrQUY5NA==" [src/protocol.rs:90] &data_bytes = [ 104, 116, 116, 112, 115, 58, 47, 47, 119, 119, 119, 46, 121, 111, 117, 116, 117, 98, 101, 46, 99, 111, 109, 47, 119, 97, 116, 99, 104, 63, 118, 61, 77, 118, 98, 108, 100, 99, 107, 65, 70, 57, 52, ] [src/protocol.rs:92] &data_string = "https://www.youtube.com/watch?v=MvbldckAF94" [cooper@archlinux Downloads]$

Here's what cargo run output looks like: `$ cargo run -- 'mpv://aHR0cHM6Ly93d3cueW91dHViZS5jb20vd2F0Y2g/dj1NdmJsZGNrQUY5NA==/?downloader=mpv&quality=best' Compiling proc-macro2 v1.0.28 Compiling unicode-xid v0.2.2 Compiling syn v1.0.75 Compiling serde_derive v1.0.129 Compiling libc v0.2.100 Compiling serde v1.0.129 Compiling base64 v0.13.0 Compiling quote v1.0.9 Compiling dirs-sys v0.3.6 Compiling dirs v3.0.2 Compiling thiserror-impl v1.0.26 Compiling thiserror v1.0.26 Compiling toml v0.5.8 Compiling mpv-handler v0.2.13 (/home/cooper/mpv-handler) warning: unreachable statement --> src/handler.rs:163:9 161 return Ok(()); ------------- any code following this expression is unreachable 162 163 / let downloader = std::process::Command::new(bin) 164 .args(downloader_options) 165 .arg(&self.protocol.url) 166 .status(); __^ unreachable statement
= note: `#[warn(unreachable_code)]` on by default
warning: unused variable: bin --> src/handler.rs:158:9 158 bin: &String, ^^^ help: if this is intentional, prefix it with an underscore: _bin
= note: `#[warn(unused_variables)]` on by default

warning: unused variable: downloader_options --> src/handler.rs:159:9 | 159 | downloader_options: Vec<&String>, | ^^^^^^^^^^^^^^^^^^ help: if this is intentional, prefix it with an underscore: _downloader_options

warning: mpv-handler (bin "mpv-handler") generated 3 warnings Finished dev [unoptimized + debuginfo] target(s) in 11.48s Running target/debug/mpv-handler 'mpv://aHR0cHM6Ly93d3cueW91dHViZS5jb20vd2F0Y2g/dj1NdmJsZGNrQUY5NA==/?downloader=mpv&quality=best' [src/protocol.rs:88] &data_base64 = "aHR0cHM6Ly93d3cueW91dHViZS5jb20vd2F0Y2g/dj1NdmJsZGNrQUY5NA==" [src/protocol.rs:90] &data_bytes = [ 104, 116, 116, 112, 115, 58, 47, 47, 119, 119, 119, 46, 121, 111, 117, 116, 117, 98, 101, 46, 99, 111, 109, 47, 119, 97, 116, 99, 104, 63, 118, 61, 77, 118, 98, 108, 100, 99, 107, 65, 70, 57, 52,] [src/protocol.rs:92] &data_string = "https://www.youtube.com/watch?v=MvbldckAF94" [cooper@archlinux mpv-handler]$`

akiirui commented 3 years ago

Oh, thanks! That output shows no problem found.

base64::decode to bytes succeed and no invalid UTF-8 sequence found:

[src/protocol.rs:90] &data_bytes = [ 104, 116, 116, 112, 115, 58, 47, 47, 119, 119, 119, 46, 121, 111, 117, 116, 117, 98, 101, 46, 99, 111, 109, 47, 119, 97, 116, 99, 104, 63, 118, 61, 77, 118, 98, 108, 100, 99, 107, 65, 70, 57, 52, ]

String::from_utf8 to string from bytes succeed:

[src/protocol.rs:92] &data_string = "https://www.youtube.com/watch?v=MvbldckAF94"

Can you try branch main? I think it works fine.

ghost commented 3 years ago

Oh, thanks! That output shows no problem found.

base64::decode to bytes succeed and no invalid UTF-8 sequence found:

[src/protocol.rs:90] &data_bytes = [ 104, 116, 116, 112, 115, 58, 47, 47, 119, 119, 119, 46, 121, 111, 117, 116, 117, 98, 101, 46, 99, 111, 109, 47, 119, 97, 116, 99, 104, 63, 118, 61, 77, 118, 98, 108, 100, 99, 107, 65, 70, 57, 52, ]

String::from_utf8 to string from bytes succeed:

[src/protocol.rs:92] &data_string = "https://www.youtube.com/watch?v=MvbldckAF94"

Can you try branch main? I think it works fine.

Yes. No problems & the stream starts playback without any problems. But the package from aur repo doesn't seem to work somehow. How to compile the binary from your source btw ? 😼

akiirui commented 3 years ago

But the package from aur repo doesn't seem to work somehow.

I think this isn't a problem with aur repo, because it's same as the main branch.

We need one more test.

ghost commented 3 years ago

But the package from aur repo doesn't seem to work somehow.

I think this isn't a problem with aur repo, because it's same as the main branch.

We need one more test.

Also can it be issue from userscript side?

akiirui commented 3 years ago

Edit YOUR_DEBUG_BINARY_PATH to the path of debug build binary

git clone https://github.com/akiirui/mpv-handler.git -b debug
cd mpv-handler
git pull
cargo build
echo "$PWD/target/debug/mpv-handler"
mkdir -p ~/.local/share/applications
cp /usr/share/applications/mpv-handler.desktop ~/.local/share/applications/
sed -i 's|Exec=mpv-handler|Exec=YOUR_DEBUG_BINARY_PATH|' ~/.local/share/applications/mpv-handler.desktop
update-desktop-database ~/.local/share/applications/

And click mpv icon on the video page in the browser.

akiirui commented 3 years ago

How to compile the binary from your source btw?

Ah, that AUR package is compiling from source. Is you install it from AUR (by paru/yay)? or another user's unofficial repository?

ghost commented 3 years ago

Edit YOUR_DEBUG_BINARY_PATH to the path of debug build binary

git clone https://github.com/akiirui/mpv-handler.git -b debug
cd mpv-handler
git pull
cargo build
echo "$PWD/target/debug/mpv-handler"
mkdir -p ~/.local/share/applications
cp /usr/share/applications/mpv-handler.desktop ~/.local/share/applications/
sed -i 's|Exec=mpv-handler|Exec=YOUR_DEBUG_BINARY_PATH|' ~/.local/share/applications/mpv-handler.desktop
update-desktop-database ~/.local/share/applications/

And click mpv icon on the video page in the browser.

Replacing exec just throws KIO error from kde dialog client Could not find the program 'YOUR_DEBUG_BINARY_PATH' And issue still persist with the default 'mpv-handler %u' Exec Also I don't think AUR package is a problem too. $ pikaur -Si mpv-handler Repository : AUR Name : mpv-handler Version : 0.2.13-1 Maintainer : akiirui URL : https://github.com/akiirui/mpv-handler/ AUR URL : https://aur.archlinux.org/packages.php?ID=951581 License : MIT Votes : 1 Popularity : 0.13% Installed : Yes Out Of Date : No Depends On : mpv youtube-dl Make Deps : cargo git Check Deps : None Optional Deps : None Provides : None Conflicts With : None Replaces : None Package Base : mpv-handler Last Update : Tue Aug 24 06:34:37 2021 Description : Play website videos and songs with mpv & youtube-dl.

akiirui commented 3 years ago

Ah, remember edit YOUR_DEBUG_BINARY_PATH in

sed -i 's|Exec=mpv-handler|Exec=YOUR_DEBUG_BINARY_PATH|' ~/.local/share/applications/mpv-handler.desktop
ghost commented 3 years ago

Ah, remember edit YOUR_DEBUG_BINARY_PATH in

sed -i 's|Exec=mpv-handler|Exec=YOUR_DEBUG_BINARY_PATH|' ~/.local/share/applications/mpv-handler.desktop

Fun fact is that the copied url from mpv-handler userscript button works perfectly fine when ran through cli. It's just the browser redirect throws this specific issue.

akiirui commented 3 years ago

That path is previous step echo "$PWD/target/debug/mpv-handler" output.

It's just the browser redirect throws this specific issue.

Yep, so we are looking at what the browser sends now.

ghost commented 3 years ago

That path is previous step echo "$PWD/target/debug/mpv-handler" output.

It's just the browser redirect throws this specific issue.

Yep, so we are looking at what the browser sends now.

Hm, the path is where I cloned debug branch. I assume there's no problem with the installed one from /usr/bin too. It must be some kind of xdg-open bug.

akiirui commented 3 years ago

The debug build will print what browser sends.

I want to know what the browser sent. please try it and paste the output to me, thanks.

ghost commented 3 years ago

Screenshot_20210921_062038

akiirui commented 3 years ago

Emm, this does not look like debug build output. Ensure the Exec path in ~/.local/share/applications/mpv-handler.desktop is the debug binary?

ghost commented 3 years ago

Emm, this does not look like debug build output. Ensure the Exec path in ~/.local/share/applications/mpv-handler.desktop is the debug binary?

I infact tried to have exec path as compiled debug binary too Exec=~/Downloads/mpv-handler/target/debug/mpv-handler %u This is it's output though. Hope it might be helpful... [src/protocol.rs:88] &data_base64 = "ahr0chm6ly93d3cuew91dhvizs5jb20vd2f0y2g/dj0yZExhdDhqOFdWMA==" [src/protocol.rs:90] &data_bytes = [ 106, 26, 244, 114, 25, 186, 151, 47, 119, 119, 119, 46, 123, 15, 117, 118, 27, 226, 206, 206, 99, 111, 109, 47, 119, 103, 244, 203, 104, 63, 118, 61, 50, 100, 76, 97, 116, 56, 106, 56, 87, 86, 48, ] Error: Failed to convert video URL string: invalid utf-8 sequence of 1 bytes from index 2

akiirui commented 3 years ago

Thanks, can you give me URL from video page -> mpv icon -> right click -> copy link address?

ghost commented 3 years ago

Thanks, can you give me URL from video page -> mpv icon -> right click -> copy link address?

mpv://aHR0cHM6Ly93d3cueW91dHViZS5jb20vd2F0Y2g/dj0yZExhdDhqOFdWMA==/?downloader=mpv&quality=best

When ran directly through cli... mpv-handler <above mpv-handler string> the stream loads fine. This must be xdg open error, that caused by the desktop shortcut from share/applications. The shortcut can't able to grab the url with right format it seems.

akiirui commented 3 years ago

Oh, thanks! Maybe I found the cause of this issue!

The origin URL:

mpv://aHR0cHM6Ly93d3cueW91dHViZS5jb20vd2F0Y2g/dj0yZExhdDhqOFdWMA==/?downloader=mpv&quality=best

The mpv-handler gets:

ahr0chm6ly93d3cuew91dhvizs5jb20vd2f0y2g/dj0yZExhdDhqOFdWMA==

The characters before the first / are all lowercase, Then base64 decoded data is different with origin data.

I cannot reproduce on GNOME xdg-open, and it's maybe caused by KIO xdg-open , it convert all "domain" part to lowercase.

I will report it to upstream, and I will push a magic to fix this problem for mpv-handler.

Finally, thank you for your help debugging!

ghost commented 3 years ago

Oh, thanks! Maybe I found the cause of this issue!

The origin URL:

mpv://aHR0cHM6Ly93d3cueW91dHViZS5jb20vd2F0Y2g/dj0yZExhdDhqOFdWMA==/?downloader=mpv&quality=best

The mpv-handler gets:

ahr0chm6ly93d3cuew91dhvizs5jb20vd2f0y2g/dj0yZExhdDhqOFdWMA==

The characters before the first / are all lowercase, Then base64 decoded data is different with origin data.

I cannot reproduce on GNOME xdg-open, and it's maybe caused by KIO xdg-open , it convert all "domain" part to lowercase.

I will report it to upstream, and I will push a magic to fix this problem for mpv-handler.

Finally, thank you for your help debugging!

Hehe no prob. I'm always available if you need more help. I'd luv to offload all online streams on mpv, caz why not...😀

akiirui commented 3 years ago

Please update mpv-handler and userscript~

ghost commented 3 years ago

Please update mpv-handler and userscript~

Error: Wrong protocol URL

Warning: Program '/home/cooper/Downloads/mpv-handler/target/debug/mpv-handler' crashed.

akiirui commented 3 years ago

Oh

Please update mpv-handler and userscript~

Error: Wrong protocol URL

Warning: Program '/home/cooper/Downloads/mpv-handler/target/debug/mpv-handler' crashed.

Oh, please remove .local/share/applications/mpv-handler.desktop, that is debug file, doesn't need anymore

ghost commented 3 years ago

Oh

Please update mpv-handler and userscript~

Error: Wrong protocol URL Warning: Program '/home/cooper/Downloads/mpv-handler/target/debug/mpv-handler' crashed.

Oh, please remove .local/share/applications/mpv-handler.desktop, that is debug file, doesn't need anymore

Previous url issue again. Above error occurs from the recompiled binary after merging latest commits.

akiirui commented 3 years ago

Don't forget upgrade usersciprt... It is updated too

latest mpv-handler: 0.2.14 userscript: 2021.09.21

ghost commented 3 years ago

Don't forget upgrade usersciprt... It is updated too

latest mpv-handler: 0.2.14 userscript: 2021.09.21

Yep. Updated both actually.

akiirui commented 3 years ago

Yep. Updated both actually.

Still Error:Wrong protocol URL`?

Please copy mpv icon address to me....

ghost commented 3 years ago

Yep. Updated both actually.

Still Error:Wrong protocol URL`?

Please copy mpv icon address to me....

cli result with url in it: mpv-handler mpv://aHR0cHM6Ly93d3cueW91dHViZS5jb20vd2F0Y2g/dj1kMFMyR3A3a2Q1OA==/?downloader=mpv&quality=best [1] 583433 [cooper@archlinux debug]$ Error: Wrong protocol URL

akiirui commented 3 years ago

Yep. Updated both actually.

Still Error:Wrong protocol URL`? Please copy mpv icon address to me....

cli result with url in it: mpv-handler mpv://aHR0cHM6Ly93d3cueW91dHViZS5jb20vd2F0Y2g/dj1kMFMyR3A3a2Q1OA==/?downloader=mpv&quality=best [1] 583433 [cooper@archlinux debug]$ Error: Wrong protocol URL

Ah, I'm sure you're not upgrade userscript. because new URL format is mpv://play/BASE64_DATA. a simple magic.

akiirui commented 3 years ago

Emm. reload the video page and move cursor over the mpv icon, see the URL?

If it starts with mpv://play/ it will work.

ghost commented 3 years ago

Yep. Updated both actually.

Still Error:Wrong protocol URL`? Please copy mpv icon address to me....

cli result with url in it: mpv-handler mpv://aHR0cHM6Ly93d3cueW91dHViZS5jb20vd2F0Y2g/dj1kMFMyR3A3a2Q1OA==/?downloader=mpv&quality=best [1] 583433 [cooper@archlinux debug]$ Error: Wrong protocol URL

Ah, I'm sure you're not upgrade userscript. because new URL format is mpv://play/BASE64_DATA. a simple magic.

Ah, I've upgraded. Just didn't restart browser. Solved now...👍

akiirui commented 3 years ago

Ah, I've upgraded. Just didn't restart browser. Solved now...+1

Okay, enjoy~ thanks for your help!