search

marcopeocchi / yt-dlp-web-ui

A terrible web ui for yt-dlp. Designed to be self-hosted.
Mozilla Public License 2.0
657 stars 67 forks source link

Download related issues discussion #146

Open CrazyWolf13 opened 2 months ago

CrazyWolf13 commented 2 months ago

Hi @marcopeocchi I really like you app and your new GUI is amazing!

Sadly I cannot download any videos at all, nor video, nor video with/without template, they appear on the gallery, but only as a url withg rey review, and after like 2 seconds, so I guess yt-dlp isn't even starting up?

also I noticed, using the :latest on docker, there is no template option when trying to download a video.

clicking the download archive without selecting anything, downloads an empty zip

also on the bottom there is an infinitely turning RPC auth circle. (currently not using any auth.)

Here the log:

Connected!
time=2024-04-17T21:53:44.929Z level=INFO msg="retrieving metadata" id=7f06de25 url="https://www.youtube.com/watch?v=t9hDbz5Q7LU"
time=2024-04-17T21:53:45.765Z level=INFO msg=finished id=7f06de25 url="https://www.youtube.com/watch?v=t9hDbz5Q7LU"
time=2024-04-17T21:54:05.593Z level=INFO msg="Trying killing process with id" id=feba1d85-4995-4370-ac6a-b0f19b6d2212
time=2024-04-17T21:54:06.433Z level=INFO msg="Trying killing process with id" id=e1d7a7ce-9313-4b77-b8e3-ff0d620ef91c
time=2024-04-17T21:54:07.388Z level=INFO msg="Trying killing process with id" id=7f06de25-3215-4475-aa0a-9270d8008dc3
time=2024-04-17T21:54:21.870Z level=INFO msg="retrieving metadata" id=863014ca url="https://www.youtube.com/watch?v=t9hDbz5Q7LU"
time=2024-04-17T21:54:22.660Z level=INFO msg=finished id=863014ca url="https://www.youtube.com/watch?v=t9hDbz5Q7LU"
time=2024-04-17T21:56:33.773Z level=INFO msg="sucessfully persisted session"
CrazyWolf13 commented 2 months ago

When clicking download or view on the card I end up at this URL: image

marcopeocchi commented 2 months ago

Hello!

Ok, this is super weird. I need to investigate. my guess is that yt-dlp from alpine packages is broken (the rpc isn't able to fetch yt-dlp version and the downloads are killed immediately).

CrazyWolf13 commented 2 months ago

Hello!

Ok, this is super weird. I need to investigate. my guess is that yt-dlp from alpine packages is broken (the rpc isn't able to fetch yt-dlp version and the downloads are killed immediately).

Yeah that makes a lot of sense! lmk, if I can test / debug anything :)

marcopeocchi commented 2 months ago

image

it is indeed broken

marcopeocchi commented 2 months ago

image

image

As expected, using an older build of yt-dlp everything works. This will be a temporary solution until the alpine:edge build of yt-dlp will be fixed.

CrazyWolf13 commented 2 months ago

Awesome!

Are you building a new img or can I just change it in the docker file and build ?

marcopeocchi commented 2 months ago

a new build is in progress https://github.com/marcopeocchi/yt-dlp-web-ui/actions/runs/8735270433

marcopeocchi commented 2 months ago

Hello @CrazyWolf13

I migrated the base image from alpine linux to Wolfi. The yt-dlp release will be always inline with the latest one on GitHub.

If there's nothing else, I think the issue can be closed :sunglasses:.

CrazyWolf13 commented 1 month ago

@marcopeocchi Awesome!

Playlist function seems to still be broken, in the logs it just shows:

Connected!
time=2024-05-05T17:01:50.166Z level=INFO msg="decoding metadata" url="https://www.youtube.com/playlist?list=PLbxxxxxYJU6-xxxxxxxxxxxxxx"

And nothing ever happens.

Issue 2:

When downloading a video and clicking "Download" the file name is like the first 6 characters and then nothing more, not even a file extension.

CrazyWolf13 commented 1 month ago

Hi @marcopeocchi

Does the playlist download work for you? It may also be a config mistake on my side.

marcopeocchi commented 1 month ago

Hi @CrazyWolf13, it does work but there's a problem with the frontend (the dialog does not close). I'll fix it by today (hopefully).

marcopeocchi commented 1 month ago

The dialog not closing has yet to be fixed. The issue was that metadata per video were not set using "donwload playlist", so they couldn't appear on the frontend. This has been fixed.

CrazyWolf13 commented 1 month ago

Hi @marcopeocchi Thanks for looking into this. Using the build published around 10h ago, the dialogue does still not close automatically, however I personally don't mind it that much.

The videos are downloading now, however custom arguments don't apply to the playlist at all.

Set the following options: image Then added a valid Template, which worked

Added the following Template and as a custom yt-dlp args as well:

-o '%(playlist_title)s/%(title)s.%(ext)s' --extract-audio --audio-format mp3

After that I got standard webm videos

marcopeocchi commented 1 month ago

Hello @CrazyWolf13,

Thanks for trying out the latest build. It seems that "playlist download" needs a little bit more work 😄. I'll take a look on this.

CrazyWolf13 commented 1 month ago

Hello @CrazyWolf13,

Thanks for trying out the latest build. It seems that "playlist download" needs a little bit more work 😄. I'll take a look on this.

Awesome, thanks!!! 🫶

marcopeocchi commented 1 month ago

8f10baf09ba2eea72a13d87cbb6ef73d10de1465 should have resolved the issue for custom arguments / template arguments in "playlist download".

CrazyWolf13 commented 1 month ago

Good evening @marcopeocchi

Sadly the issue still persists and everything behaves exactly the same. Can you reproduce the issue, with the above metrics?

marcopeocchi commented 1 month ago

Hello @CrazyWolf13,

I've reproduced the issue with the arguments you provide and it seems working :/

image

image

image

Also, enabling "custom yt-dlp arguments" in conjunction with a "custom template" behaves prefectly fine

CrazyWolf13 commented 1 month ago

Hi @marcopeocchi

Thanks, I'll look into this in the evening.

One thing I noticed, your link is a youtube watch url not a playlist url, I'm a bit confused?

Normal video url: https://youtube.com/watch?v=pWZAskDksjg

Normal playlist url: https://youtube.com/playlist?list=PLRBp0Fe2GpglJoiVdv1UjArQHn2veCqst

But you seem to have used a video url with a download playlist argument?

CrazyWolf13 commented 1 month ago

Maybe it would be good to have access to the yt-dlp logs, are they saved anywhere inside the container, or can they be displayed on the logs section?

Because currently I see only an entry:

Connected!
time=2024-05-15T14:56:31.614Z level=INFO msg="decoding playlist metadata" url="{link to my playlist}"
marcopeocchi commented 1 month ago

Hi @CrazyWolf13,

I've tried the playlist url you provided on the latter comment. Here's the log:

time=2024-05-16T10:05:19.727+02:00 level=INFO msg="yt-dlp-webui started" address=0.0.0.0:3033
time=2024-05-16T10:05:19.728+02:00 level=INFO msg="sucessfully persisted session"
time=2024-05-16T10:06:04.883+02:00 level=INFO msg="decoding playlist metadata" url="https://youtube.com/playlist?list=PLRBp0Fe2GpglJoiVdv1UjArQHn2veCqst"
time=2024-05-16T10:06:26.170+02:00 level=INFO msg="decoded metadata" url="https://youtube.com/playlist?list=PLRBp0Fe2GpglJoiVdv1UjArQHn2veCqst"
time=2024-05-16T10:06:26.170+02:00 level=INFO msg="playlist detected" url="https://youtube.com/playlist?list=PLRBp0Fe2GpglJoiVdv1UjArQHn2veCqst" count=11
time=2024-05-16T10:06:26.172+02:00 level=INFO msg="received process from event bus" bus=process:pending consumer=metadataConsumer id=4bcb795a-dcfd-4e43-b43f-f68e8de45f1c
time=2024-05-16T10:06:26.172+02:00 level=INFO msg="received process from event bus" bus=process:pending consumer=downloadConsumer id=4bcb795a-dcfd-4e43-b43f-f68e8de45f1c
time=2024-05-16T10:06:26.174+02:00 level=INFO msg="retrieving metadata" id=4bcb795a url=""
time=2024-05-16T10:06:36.369+02:00 level=INFO msg=finished id=4bcb795a url=""
time=2024-05-16T10:06:36.369+02:00 level=INFO msg="started process" bus=process:pending id=4bcb795a-dcfd-4e43-b43f-f68e8de45f1c
time=2024-05-16T10:06:36.391+02:00 level=ERROR msg="failed to retrieve metadata" id=4bcb795a-dcfd-4e43-b43f-f68e8de45f1c err="ERROR: [generic] '' is not a valid URL. Set --default-search \"ytsearch\" (or run  yt-dlp \"ytsearch:\" ) to search YouTube\n"
time=2024-05-16T10:06:36.392+02:00 level=INFO msg="received process from event bus" bus=process:pending consumer=metadataConsumer id=a100ab80-5e32-41f3-adf5-93eab2bb465d
time=2024-05-16T10:06:36.392+02:00 level=INFO msg="received process from event bus" bus=process:pending consumer=downloadConsumer id=a100ab80-5e32-41f3-adf5-93eab2bb465d
time=2024-05-16T10:06:36.394+02:00 level=INFO msg="retrieving metadata" id=a100ab80 url="https://www.youtube.com/watch?v=GK-u7_mxum8"
time=2024-05-16T10:06:48.740+02:00 level=INFO msg="received process from event bus" bus=process:pending consumer=metadataConsumer id=22b2a392-52a8-4ba3-979b-05218f8c9303
time=2024-05-16T10:06:48.740+02:00 level=INFO msg="received process from event bus" bus=process:pending consumer=downloadConsumer id=22b2a392-52a8-4ba3-979b-05218f8c9303
time=2024-05-16T10:06:48.741+02:00 level=INFO msg="retrieving metadata" id=22b2a392 url="https://www.youtube.com/watch?v=MwYWnzGsdSM"
time=2024-05-16T10:06:48.894+02:00 level=INFO msg=progress id=a100ab80 url="https://www.youtube.com/watch?v=GK-u7_mxum8" percentage="  2.7%"
time=2024-05-16T10:06:49.395+02:00 level=INFO msg=progress id=a100ab80 url="https://www.youtube.com/watch?v=GK-u7_mxum8" percentage=" 10.9%"
time=2024-05-16T10:06:50.302+02:00 level=INFO msg=progress id=a100ab80 url="https://www.youtube.com/watch?v=GK-u7_mxum8" percentage=" 43.5%"
time=2024-05-16T10:06:50.395+02:00 level=INFO msg=progress id=a100ab80 url="https://www.youtube.com/watch?v=GK-u7_mxum8" percentage=" 43.5%"
time=2024-05-16T10:06:50.894+02:00 level=INFO msg=progress id=a100ab80 url="https://www.youtube.com/watch?v=GK-u7_mxum8" percentage=" 56.4%"
time=2024-05-16T10:06:51.395+02:00 level=INFO msg=progress id=a100ab80 url="https://www.youtube.com/watch?v=GK-u7_mxum8" percentage=" 64.6%"
time=2024-05-16T10:06:52.319+02:00 level=INFO msg=progress id=a100ab80 url="https://www.youtube.com/watch?v=GK-u7_mxum8" percentage=" 97.2%"
time=2024-05-16T10:06:52.395+02:00 level=INFO msg=progress id=a100ab80 url="https://www.youtube.com/watch?v=GK-u7_mxum8" percentage=" 97.2%"
time=2024-05-16T10:06:53.115+02:00 level=INFO msg="started process" bus=process:pending id=a100ab80-5e32-41f3-adf5-93eab2bb465d
time=2024-05-16T10:06:53.115+02:00 level=INFO msg=finished id=a100ab80 url="https://www.youtube.com/watch?v=GK-u7_mxum8"
time=2024-05-16T10:07:01.377+02:00 level=INFO msg="received process from event bus" bus=process:pending consumer=metadataConsumer id=e2cf4e75-5bef-43be-b0f1-dee4ddac8129
time=2024-05-16T10:07:01.377+02:00 level=INFO msg="received process from event bus" bus=process:pending consumer=downloadConsumer id=e2cf4e75-5bef-43be-b0f1-dee4ddac8129
time=2024-05-16T10:07:01.379+02:00 level=INFO msg="retrieving metadata" id=e2cf4e75 url="https://www.youtube.com/watch?v=Zx8D2bRB2OM"
time=2024-05-16T10:07:01.742+02:00 level=INFO msg=progress id=22b2a392 url="https://www.youtube.com/watch?v=MwYWnzGsdSM" percentage="  2.3%"
time=2024-05-16T10:07:02.242+02:00 level=INFO msg=progress id=22b2a392 url="https://www.youtube.com/watch?v=MwYWnzGsdSM" percentage="  9.2%"
time=2024-05-16T10:07:03.112+02:00 level=INFO msg=progress id=22b2a392 url="https://www.youtube.com/watch?v=MwYWnzGsdSM" percentage=" 36.7%"
time=2024-05-16T10:07:03.242+02:00 level=INFO msg=progress id=22b2a392 url="https://www.youtube.com/watch?v=MwYWnzGsdSM" percentage=" 36.7%"
time=2024-05-16T10:07:03.921+02:00 level=INFO msg=progress id=22b2a392 url="https://www.youtube.com/watch?v=MwYWnzGsdSM" percentage=" 52.9%"
time=2024-05-16T10:07:04.242+02:00 level=INFO msg=progress id=22b2a392 url="https://www.youtube.com/watch?v=MwYWnzGsdSM" percentage=" 52.9%"
time=2024-05-16T10:07:05.063+02:00 level=INFO msg=progress id=22b2a392 url="https://www.youtube.com/watch?v=MwYWnzGsdSM" percentage=" 80.5%"
time=2024-05-16T10:07:05.242+02:00 level=INFO msg=progress id=22b2a392 url="https://www.youtube.com/watch?v=MwYWnzGsdSM" percentage=" 80.5%"
time=2024-05-16T10:07:05.886+02:00 level=INFO msg=progress id=22b2a392 url="https://www.youtube.com/watch?v=MwYWnzGsdSM" percentage=" 98.7%"
time=2024-05-16T10:07:06.358+02:00 level=INFO msg="started process" bus=process:pending id=22b2a392-52a8-4ba3-979b-05218f8c9303
time=2024-05-16T10:07:06.358+02:00 level=INFO msg=finished id=22b2a392 url="https://www.youtube.com/watch?v=MwYWnzGsdSM"
time=2024-05-16T10:07:14.186+02:00 level=INFO msg="received process from event bus" bus=process:pending consumer=metadataConsumer id=04da11b3-2503-4f6f-9c0f-a35f58ddfb4d
time=2024-05-16T10:07:14.186+02:00 level=INFO msg="received process from event bus" bus=process:pending consumer=downloadConsumer id=04da11b3-2503-4f6f-9c0f-a35f58ddfb4d
time=2024-05-16T10:07:14.188+02:00 level=INFO msg="retrieving metadata" id=04da11b3 url=""
time=2024-05-16T10:07:14.987+02:00 level=INFO msg=progress id=e2cf4e75 url="https://www.youtube.com/watch?v=Zx8D2bRB2OM" percentage=" 11.0%"
time=2024-05-16T10:07:16.130+02:00 level=INFO msg=progress id=e2cf4e75 url="https://www.youtube.com/watch?v=Zx8D2bRB2OM" percentage=" 44.2%"

Note: time=2024-05-16T10:06:04.883+02:00 level=INFO msg="decoding playlist metadata" url="https://youtube.com/playlist?list=PLRBp0Fe2GpglJoiVdv1UjArQHn2veCqst" took a solid 30-40 seconds in my case.

Eveything seems working as intended :/ This is the template I used: -o '%(playlist_title)s/%(title)s.%(ext)s' --extract-audio --audio-format opus, which is analogous to yours.

Here's the output: image

Let me know if in the latest latest build this happens again.

CrazyWolf13 commented 1 month ago

Apologies for the really late reply, I've tested a lot of things on my end to ensure nothing is interfering with the actual yt-dlp processs, like for example pi-hole blocking or similar.

For me it does still not work, i'm getting the following error:

time=2024-05-21T12:18:05.742Z level=INFO msg="yt-dlp-webui started" address=0.0.0.0:3033
time=2024-05-21T12:19:12.671Z level=INFO msg="decoding playlist metadata" url="https://www.youtube.com/playlist?list=PLbPRSq5YJU6-D8YXpoU0zQzU5Ml8mrGGy"
2024/05/21 12:20:32 http: response.WriteHeader on hijacked connection from github.com/marcopeocchi/yt-dlp-web-ui/server/rpc.WebSocket (handlers.go:33)
2024/05/21 12:20:32 http: response.Write on hijacked connection from fmt.Fprintln (print.go:305)
2024/05/21 12:20:32 websocket: close 1001 (going away)
time=2024-05-21T12:21:13.933Z level=INFO msg="decoding playlist metadata" url="https://www.youtube.com/playlist?list=PLbPRSq5YJU6-D8YXpoU0zQzU5Ml8mrGGy"
2024/05/21 12:22:54 http: response.WriteHeader on hijacked connection from github.com/marcopeocchi/yt-dlp-web-ui/server/rpc.WebSocket (handlers.go:33)
2024/05/21 12:22:54 http: response.Write on hijacked connection from fmt.Fprintln (print.go:305)
2024/05/21 12:22:54 websocket: close 1006 (abnormal closure): unexpected EOF
time=2024-05-21T12:23:05.742Z level=INFO msg="sucessfully persisted session"

Btw, is yt-dlp-webui using the android client?

The android client for metadata got droppend in some of the latest releases.

CrazyWolf13 commented 1 month ago

@marcopeocchi After trying today again, I get the following errors in the console and a the same error visualy shown on the webpage: image

Hope this helps and huge thanks for looking into this.

marcopeocchi commented 1 month ago

I'll look into. It might be more complicated than it looks 🥲.

CrazyWolf13 commented 1 month ago

Huge thanks!

Otherwise it would maybe be nice to have some basic checks, e.g. is internet connection to the outside alive, dns resolving, all youtube domains reachable etc.

Also maybe more yt-dlp log would make it easier for people to diagnose this.

CrazyWolf13 commented 2 weeks ago

hi @marcopeocchi New release seems to have fixed the bug where the window would stay open after submitting a playlist.

using the default profile seemed to kinda start a download process, however it took very long and I've got very many erros. Initially started the command at 18:36 ended nearly 20 minutes later. for 16videos or so, which normally barely take over 2minutes on yt-dlp. Also there were only 2 files effectively written to the disk from those 16 or so.

Using my template as specified above does still result in nothing, but decoding metadata.

time=2024-06-12T18:48:43.001Z level=ERROR msg="failed to retrieve metadata" id=18c6b955 err="WARNING: [youtube] Unable to download webpage: [Errno -3] Try again\nWARNING: [youtube] [Errno -3] Try again. Retrying (1/3)...\nWARNING: [youtube] [Errno -3] Try again. Retrying (2/3)...\nWARNING: [youtube] [Errno -3] Try again. Retrying (3/3)...\nWARNING: [youtube] Unable to download API page: [Errno -3] Try again (caused by TransportError('[Errno -3] Try again'))\nWARNING: [youtube] Unable to download webpage: [Errno -3] Try again\nWARNING: [youtube] [Errno -3] Try again. Retrying (1/3)...\nWARNING: [youtube] [Errno -3] Try again. Retrying (2/3)...\nWARNING: Only images are available for download. use --list-formats to see them\nERROR: [youtube] lHZtcC67yrY: Requested format is not available. Use --list-formats for a list of available formats\n"
time=2024-06-12T18:48:43.001Z level=INFO msg="received process from event bus" bus=process:pending consumer=metadataConsumer id=af835bbc
time=2024-06-12T18:48:43.001Z level=INFO msg="retrieving metadata" id=af835bbc url="https://www.youtube.com/watch?v=3MSO7rUcRsY"
time=2024-06-12T18:49:10.524Z level=INFO msg="received process from event bus" bus=process:pending consumer=metadataConsumer id=f0e04d6f
time=2024-06-12T18:49:10.524Z level=INFO msg="retrieving metadata" id=f0e04d6f url="https://www.youtube.com/watch?v=Vfgk3gt5e6A"
time=2024-06-12T18:51:13.418Z level=INFO msg="received process from event bus" bus=process:pending consumer=metadataConsumer id=1c23093e
time=2024-06-12T18:51:13.418Z level=INFO msg="retrieving metadata" id=1c23093e url="https://www.youtube.com/watch?v=UcCmJba6ERA"
time=2024-06-12T18:51:24.566Z level=INFO msg=progress id=6f15d51d url="https://www.youtube.com/watch?v=2Yezroz4O7M" percentage=" 93.3%"
time=2024-06-12T18:51:45.296Z level=INFO msg="started process" bus=process:pending id=6f15d51d
time=2024-06-12T18:51:45.296Z level=INFO msg=finished id=6f15d51d url="https://www.youtube.com/watch?v=2Yezroz4O7M"
time=2024-06-12T18:51:46.495Z level=INFO msg="started process" bus=process:pending id=33b7c05f
time=2024-06-12T18:51:46.495Z level=INFO msg=finished id=33b7c05f url="https://www.youtube.com/watch?v=RbVFlOTzPus"
time=2024-06-12T18:52:04.147Z level=ERROR msg="failed to retrieve metadata" id=1c23093e err="WARNING: [youtube] Unable to download webpage: [Errno -3] Try again\nWARNING: [youtube] [Errno -3] Try again. Retrying (1/3)...\nWARNING: [youtube] [Errno -3] Try again. Retrying (2/3)...\nWARNING: [youtube] [Errno -3] Try again. Retrying (3/3)...\nWARNING: [youtube] Unable to download API page: [Errno -3] Try again (caused by TransportError('[Errno -3] Try again'))\nWARNING: [youtube] Unable to download webpage: [Errno -3] Try again\nWARNING: [youtube] [Errno -3] Try again. Retrying (1/3)...\nWARNING: [youtube] [Errno -3] Try again. Retrying (2/3)...\nWARNING: [youtube] [Errno -3] Try again. Retrying (3/3)...\nWARNING: [youtube] Unable to download API page: [Errno -3] Try again (caused by TransportError('[Errno -3] Try again'))\nERROR: [youtube] UcCmJba6ERA: Failed to extract any player response; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U\n"
time=2024-06-12T18:52:04.147Z level=INFO msg="received process from event bus" bus=process:pending consumer=metadataConsumer id=6f15d51d
time=2024-06-12T18:52:04.148Z level=INFO msg="retrieving metadata" id=6f15d51d url="https://www.youtube.com/watch?v=2Yezroz4O7M"
time=2024-06-12T18:52:12.100Z level=INFO msg="sucessfully persisted session"
time=2024-06-12T18:52:24.668Z level=INFO msg=finished id=29f4159c url="https://www.youtube.com/watch?v=HB_1_rdhets"
time=2024-06-12T18:52:24.668Z level=INFO msg="started process" bus=process:pending id=29f4159c
time=2024-06-12T18:53:02.976Z level=INFO msg="started process" bus=process:pending id=839714b8
time=2024-06-12T18:53:29.630Z level=INFO msg="received process from event bus" bus=process:pending consumer=metadataConsumer id=33b7c05f
time=2024-06-12T18:53:29.631Z level=INFO msg="retrieving metadata" id=33b7c05f url="https://www.youtube.com/watch?v=RbVFlOTzPus"
time=2024-06-12T18:53:57.039Z level=INFO msg="received process from event bus" bus=process:pending consumer=metadataConsumer id=29f4159c
time=2024-06-12T18:53:57.040Z level=INFO msg="retrieving metadata" id=29f4159c url="https://www.youtube.com/watch?v=HB_1_rdhets"
time=2024-06-12T18:54:04.990Z level=INFO msg="received process from event bus" bus=process:pending consumer=metadataConsumer id=839714b8
time=2024-06-12T18:54:04.991Z level=INFO msg="retrieving metadata" id=839714b8 url="https://www.youtube.com/watch?v=6ur2UpD3Mwc"
time=2024-06-12T18:54:06.048Z level=INFO msg=finished id=e87ff8a3 url="https://www.youtube.com/playlist?list=PLCvX74b4S-GssF5u-Zbud6jlt6cFnl-WX"
time=2024-06-12T18:54:52.431Z level=INFO msg="received process from event bus" bus=process:pending consumer=metadataConsumer id=e87ff8a3