nfriedly / node-unblocker

Web proxy for evading internet censorship, and general-purpose Node.js library for proxying and rewriting remote webpages
https://www.npmjs.com/package/unblocker
GNU Affero General Public License v3.0
458 stars 914 forks source link

https://www.youtube.com/embed/XGxIE1hr0w4 --This type of URL cannot be proxied #257

Closed MassSmith closed 5 months ago

MassSmith commented 5 months ago

node-unblocker is a great project, I love it, I have been using it for more than 6 years. Recently I tried the youtube proxy function, it is very useful and smooth, but one of the youtube url formats does not seem to work properly, for example: https://www.youtube.com/embed/XGxIE1hr0w4 and this format is very common, many videos embedded on web pages are in this URL format. I really hope to add this function, thank the project author for his great contribution. (My English is very poor, the above is translated by a translator)

MassSmith commented 5 months ago

Another small bug was found https://****.***.***/proxy/https://www.youtube.com/watch?v=XGxIE1hr0w4 In areas where youtube cannot be accessed, the cover image cannot be displayed. Here is the webpage source code:

071416
nfriedly commented 5 months ago

Aah, good points. For the first one, you might just need to change this line to include /embed urls:

https://github.com/nfriedly/node-unblocker/blob/1f5bdf3eb46be30501450f81165ef99f2dc9130b/examples/youtube/youtube.js#L8

-  if (hostname === "www.youtube.com" && pathname === "/watch") {
+  if (hostname === "www.youtube.com" && (pathname === "/watch" || pathname === "/embed")) {

For the second one, I think you need to add /proxy/ to the start of the poster url here:

https://github.com/nfriedly/node-unblocker/blob/1f5bdf3eb46be30501450f81165ef99f2dc9130b/examples/youtube/youtube.js#L30

- <video controls poster="${thumb.url}" style="width: 100%">
+ <video controls poster="/proxy/${thumb.url}" style="width: 100%">

Also, I think we should add poster= to this regex, although I don't think it affects the youtube page

https://github.com/nfriedly/node-unblocker/blob/1f5bdf3eb46be30501450f81165ef99f2dc9130b/lib/url-prefixer.js#L24

I don't have time to test these out right now, but that's my best guess as to the fixes; feel free to try them out if you have the time.

MassSmith commented 5 months ago

Thank you very much for helping me in your busy schedule. I have done what you told me and the cover image problem has been solved. But the first problem still exists. The corresponding source code has been changed to: “if (hostname === "www.youtube.com" && (pathname === "/watch" || pathname === "/embed")) {” But when proxying a URL in the format of https://www.youtube.com/embed/XGxIE1hr0w4, it cannot be played. Here are the test results:

205231 205700

Thank you again! !

MassSmith commented 5 months ago

In addition, there is no error output on the console。

211317
nfriedly commented 5 months ago

Oh, I see. For /watch, the video id is in the querystring, but for /embed it's in the path, so my check wouldn't work.

Instead of trying to guess whether or not it's a youtube URL ourselves, we should just ask ytdl. And, come to think of it, we could drop the first part and make this page work for other domains like youtu.be and m.youtube.com:

Can you try this and let me know if it works?

-  if (hostname === "www.youtube.com" && (pathname === "/watch" || pathname === "/embed")) {
+  if (ytdl.validateURL(data.url)) {
MassSmith commented 5 months ago

Oh, I see. For /watch, the video id is in the querystring, but for /embed it's in the path, so my check wouldn't work.

Instead of trying to guess whether or not it's a youtube URL ourselves, we should just ask ytdl. And, come to think of it, we could drop the first part and make this page work for other domains like youtu.be and m.youtube.com:

Can you try this and let me know if it works?

-  if (hostname === "www.youtube.com" && (pathname === "/watch" || pathname === "/embed")) {
+  if (ytdl.validateURL(data.url)) {

Really Great!!! Works perfectly!!! This problem is perfectly solved!! Thank you again!!

nfriedly commented 5 months ago

I went ahead and incorporated the changes so that everyone else will get it going forward. Thanks for testing things for me!

MassSmith commented 5 months ago

Thank you so much for everything you do!! ^_^