Automatically monitor youtube updates and download the latest videos

[PDSCi996]

First of all, thank you very much for your hard work. I have some bloggers that I follow frequently on youtube. I need to download their latest videos to the local. I heard that flexget can monitor and push, so I installed it on my Synology docker. , But I don’t know why the setting has been unsuccessful. Could you please help me find the suitable reason? Here is my configuration.

web_server: bind: 0.0.0.0 port: 5050 web_ui: yes

tasks: get-youtubemusic: priority: 4 rss: url: https://www.youtube.com/feeds/videos.xml?channel_id=UCUXi5mmqbvIithAs9AaxEtw accept_all: yes exec: auto_escape: yes on_output: for_accepted: 'curl -X POST --data-urlencode "url={{url}}" http://192.168.1.234:8080

http://192.168.1.234: 8080 is my youtube-dl container address

[nickv2002]

@PDSCi996 here's how I accomplish what you want to do

TWL-DL-Server - Which is a fork of this project with only minor changes to scan for videos on ToWatchList.

ToWatchList a free web service I wrote to track your Watch Later videos separately and more openly than YouTube.

Pipedream - A free web service that can run small blocks of code to connect thousands of apps. It's the glue that brings ToWatchList and YouTube together.

You can get started with this workflow I wrote that will add new videos from a YouTube playlist to ToWatchList. Here's another example that will scan an RSS feed from Kottke.org, filter out posts we don't want, and finally import any embedded videos into ToWatchList. You could easily customize it to point to a different RSS feed of your choice.

A final tip, if you'd like to import all your YouTube subscriptions into ToWatchList instead of just a playlist, you can combine the Pipedream workflow that adds new playlist items to ToWatchList with this GitHub Project: auto-youtube-subscription-playlist-2. It can also do neat things like combine multiple playlists into one playlist (which you can then import to ToWatchList).

So there you go, the end result is that when new videos are posted they are shortly downloaded by a Docker image. Takes a bit of set up but I'm happy to help with any questions.

[nbr23]

Hi @PDSCi996,

I have never used flexget, but it looks like it could do the trick. Are you getting any logs that might help us in here either from flexget or from the youtube-dl-server container ?

At first glance, the target container url seems wrong, and should be: http://192.168.1.234:8080/api/downloads to point to the correct end point. Perhaps try this edit and see if it works better.

If not, please do share logs.

Thanks!

[nickv2002]

@PDSCi996 thinking about this more, if you don't need the Watched/Favorite/App/etc you could cut ToWatchList & Pipedream out of the process by writing a 1-line script/cron to regularly download a YT playlist (optionally created by auto-youtube-subscription-playlist-2) with YouTube-DL-Server.

[PDSCi996]

@PDSCi996 here's how I accomplish what you want to do

TWL-DL-Server - Which is a fork of this project with only minor changes to scan for videos on ToWatchList.

ToWatchList a free web service I wrote to track your Watch Later videos separately and more openly than YouTube.

Pipedream - A free web service that can run small blocks of code to connect thousands of apps. It's the glue that brings ToWatchList and YouTube together.

You can get started with this workflow I wrote that will add new videos from a YouTube playlist to ToWatchList. Here's another example that will scan an RSS feed from Kottke.org, filter out posts we don't want, and finally import any embedded videos into ToWatchList. You could easily customize it to point to a different RSS feed of your choice.

A final tip, if you'd like to import all your YouTube subscriptions into ToWatchList instead of just a playlist, you can combine the Pipedream workflow that adds new playlist items to ToWatchList with this GitHub Project: auto-youtube-subscription-playlist-2. It can also do neat things like combine multiple playlists into one playlist (which you can then import to ToWatchList).

So there you go, the end result is that when new videos are posted they are shortly downloaded by a Docker image. Takes a bit of set up but I'm happy to help with any questions.

Thank you very much for your selfless help. I will study the website you gave me carefully.

[PDSCi996]

Hi @PDSCi996,

I have never used flexget, but it looks like it could do the trick. Are you getting any logs that might help us in here either from flexget or from the youtube-dl-server container ?

At first glance, the target container url seems wrong, and should be: http://192.168.1.234:8080/api/downloads to point to the correct end point. Perhaps try this edit and see if it works better.

If not, please do share logs.

Thanks!

Thank you very much, my flexget has the latest log showing "Stdout: /bin/sh: curl: not found" My system uses synology, and I directly used the image of https://registry.hub.docker.com/r/wiserain/flexget/ in the above docker suite. I checked the docker image and found that there is no curl command in it. , My preliminary judgment is that the author did not encapsulate this command, which made the command invalid. Can you teach me how to encapsulate the curl command into a docker container, or install the curl command in an already running docker container. Greatful.

[PDSCi996]

Hi @PDSCi996,

I have never used flexget, but it looks like it could do the trick. Are you getting any logs that might help us in here either from flexget or from the youtube-dl-server container ?

At first glance, the target container url seems wrong, and should be: http://192.168.1.234:8080/api/downloads to point to the correct end point. Perhaps try this edit and see if it works better.

If not, please do share logs.

Thanks!

Thank you very much, my flexget has the latest log showing "Stdout: /bin/sh: curl: not found" My system uses synology, and I directly used the image of https://registry.hub.docker.com/r/wiserain/flexget/ in the above docker suite. I checked the docker image and found that there is no curl command in it. , My preliminary judgment is that the author did not encapsulate this command, which made the command invalid. Can you teach me how to encapsulate the curl command into a docker container, or install the curl command in an already running docker container. Greatful.

[nbr23]

@PDSCi996 something like this should do: https://github.com/nbr23/docker-flexget/commit/d197ca2187667d4fd08107018429f05fa821cd03

[PDSCi996]

@PDSCi996 something like this should do: nbr23/docker-flexget@d197ca2

Thank you very much for your hard work. I have seen that you have added two commands to the original container. I just don’t know how to use the new container you changed. I downloaded the modified file and uploaded it to Synology docker manually. , The prompt is invalid, and then I don’t know what to do.

[nbr23]

@PDSCi996 I am not sure how synology handles docker, but you would need to build it first. You can get it built from here otherwise: https://registry.hub.docker.com/r/nbr23/flexget/

[PDSCi996]

@PDSCi996 I am not sure how synology handles docker, but you would need to build it first. You can get it built from here otherwise: https://registry.hub.docker.com/r/nbr23/flexget/

Thank you for your prompt reply. I have solved the problem of flexget. The rss information of youtube has been captured and pushed to youtune-dl, but the link address is all "https://www.youtube.com/watch?v=xZS- "by-7gV4", the log prompts "Error during download task", I click retry, the log prompts "ERROR: requested format not available", what's the matter, is it my configuration problem?

[nbr23]

Do you know which format it tries to download?

[PDSCi996]

Do you know which format it tries to download?

I installed one of your branches in my docker "https://github.com/ToWatchList/twl-dl-server", it made a little change based on yours, I used flexget to push to these two at the same time If the container has exactly the same parameters, this branch can be downloaded normally, but your container will prompt an error, and you don't understand the problem.

[nbr23]

I'm not sure I follow. Can you give me the video you are trying to download, which format you are asking, and the full curl command used (or flexget config) ?

Thanks!

[PDSCi996]

I'm not sure I follow. Can you give me the video you are trying to download, which format you are asking, and the full curl command used (or flexget config) ?

Thanks!

Okay, I use the container "https://registry.hub.docker.com/r/wiserain/flexget/", and the author has replied to the solution on how to install the curl command "https://github.com /wiserain/docker-flexget/issues/21", after I installed the curl command, I restarted the docker container and it can be pushed normally. The following is my flexget configuration. I have followed 5 youtube channels in total. `web_server: bind: 0.0.0.0 port: 5050 web_ui: yes

tasks: get-youtubemusic: priority: 4 rss: url: https://www.youtube.com/feeds/videos.xml?channel_id=UCUXi5mmqbvIithAs9AaxEtw accept_all: yes exec: auto_escape: yes on_output: for_accepted: 'curl -X POST --data-urlencode "url={{url}}" http://192.168.1.234:8080/api/downloads'

get-youtube: priority: 4 rss: url: https://www.youtube.com/feeds/videos.xml?channel_id=UCAk-SBMZp79a4k2CAqqoDLQ accept_all: yes exec: auto_escape: yes on_output: for_accepted: 'curl -X POST --data-urlencode "url={{url}}" http://182.168.1.234:8080/api/downloads'

get-you: priority: 4 rss: url: https://www.youtube.com/feeds/videos.xml?channel_id=UCcyq3Dtsa-EbuIfwViWbT4Q accept_all: yes exec: auto_escape: yes on_output: for_accepted: 'curl -X POST --data-urlencode "url={{url}}" http://192.168.1.234:8080/api/downloads'

jiang_jun_lai_le_8110: priority: 4 rss: url: https://www.youtube.com/feeds/videos.xml?channel_id=UCzpCqCt4NHZU2B3YBeadxng accept_all: yes exec: auto_escape: yes on_output: for_accepted: 'curl -X POST --data-urlencode "url={{url}}" http://192.168.1.234:8080/api/downloads'

icdi_8110: priority: 4 rss: url: https://www.youtube.com/feeds/videos.xml?channel_id=UCWcNljhA3iWjRMzQRuCAepw accept_all: yes exec: auto_escape: yes on_output: for_accepted: 'curl -X POST --data-urlencode "url={{url}}" http://192.168.1.234:8080/api/downloads'`

This is the specific configuration of "config-yml" under my youtube-dl container

`ydl_server: # youtube-dl-server specific settings port: 8080 # Port youtube-dl-server should listen on host: 0.0.0.0 # IP youtube-dl-server should bind to debug: False # Enable/Disable debug mode metadata_db_path: '/youtube-dl/.ydl-metadata.db' # Path to metadata DB output_playlist: '/youtube-dl/%(playlist)s/%(playlist_index)s - %(title)s.%(ext)s' # Playlist output directory update_poll_delay_min: 1440 # Automatically check for updates every 24h

ydl_options: # youtube-dl options output: '/youtube-dl/%(uploader)s/%(title)s.%(ext)s' # output directory cache-dir: '/youtube-dl/.cache' # youtube-dl cache directory ignore-errors: True # instruct youtube-dl to skip errors age-limit: 6 # minimal age requirement / parental control setting format: bestvideo[ext=mp4][height<=1080]+bestaudio[ext=m4a]/best[ext=mp4]/best`

sorry to bother you.

[PDSCi996]

@PDSCi996 here's how I accomplish what you want to do

TWL-DL-Server - Which is a fork of this project with only minor changes to scan for videos on ToWatchList.

ToWatchList a free web service I wrote to track your Watch Later videos separately and more openly than YouTube.

Pipedream - A free web service that can run small blocks of code to connect thousands of apps. It's the glue that brings ToWatchList and YouTube together.

You can get started with this workflow I wrote that will add new videos from a YouTube playlist to ToWatchList. Here's another example that will scan an RSS feed from Kottke.org, filter out posts we don't want, and finally import any embedded videos into ToWatchList. You could easily customize it to point to a different RSS feed of your choice.

A final tip, if you'd like to import all your YouTube subscriptions into ToWatchList instead of just a playlist, you can combine the Pipedream workflow that adds new playlist items to ToWatchList with this GitHub Project: auto-youtube-subscription-playlist-2. It can also do neat things like combine multiple playlists into one playlist (which you can then import to ToWatchList).

So there you go, the end result is that when new videos are posted they are shortly downloaded by a Docker image. Takes a bit of set up but I'm happy to help with any questions.

First of all, thank you very much for your hard work. I saw that the "TWL-DL-Server" container was improved on the basis of "nbr23". When I set up, I created a "/config" path like nbr23. I thought it would be generated under this path. A "config.yml" configuration file, but I found no. I am using "flexget" to push the youtube address for automatic download. I hope that the youtube-dl parameter is set to Mp4 format not greater than 1080P, and the downloaded file is automatically Create directory classification, in nbr23, set it in "config.yml". Is there a specific configuration file for your container? Thanks a lot.

[nickv2002]

First of all: TWL-DL-Server is a fork that adds integration with ToWatchList.com and Kodi (via NFO files). It's a bit behind all the recent work that @nbr23 has put into the latest branch here on the latest branch so it still uses the "Legacy Config" (legacy being current from a month or so ago 😆). So if you aren't using the Kodi or ToWatchList features, it might be better to stick to latest version on this repo.

If you do use TW-DL-Server, the easiest way to change stuff is to add environment variable to the container that overwrite the defaults in config.py since it predates the addition of a config.yml that @nbr23 added recently.

In either case I'm not sure why you need the RSS feed and Flexget, though with some scripting that will work. Instead you could just pass a whole playlist instead of individual item, youtube-dl is smart enough to download a whole playlist (and only grab new items that don't exist already), so you'll end up with the newest items if you just process the playlist regularly.

[jim60105]

@PDSCi996 I think using IFTTT to connect Youtube → Webhooks should be more suitable than RSS monitor.

In addition, if you want to use RSS feeds and monitor locally, there are simpler services that can be used, such as this. I haven't actually tested this image, but I can find a bunch of implementations with searching "rss trigger docker".

[PDSCi996]

First of all: TWL-DL-Server is a fork that adds integration with ToWatchList.com and Kodi (via NFO files). It's a bit behind all the recent work that @nbr23 has put into the latest branch here on the latest branch so it still uses the "Legacy Config" (legacy being current from a month or so ago ). So if you aren't using the Kodi or ToWatchList features, it might be better to stick to latest version on this repo.

If you do use TW-DL-Server, the easiest way to change stuff is to add environment variable to the container that overwrite the defaults in config.py since it predates the addition of a config.yml that @nbr23 added recently.

In either case I'm not sure why you need the RSS feed and Flexget, though with some scripting that will work. Instead you could just pass a whole playlist instead of individual item, youtube-dl is smart enough to download a whole playlist (and only grab new items that don't exist already), so you'll end up with the newest items if you just process the playlist regularly.

Thank you very much for your prompt reply. I have now configured the file and can download the video according to my requirements. But there is a small problem. I accidentally added a playlist with thousands of video files. How can I stop the download or delete it from the waiting sequence?

[PDSCi996]

I'm not sure I follow. Can you give me the video you are trying to download, which format you are asking, and the full curl command used (or flexget config) ?

Thanks!

I checked the log today and the following error message appeared。

172.17.0.1 - - [02/Feb/2021 07:04:46] "GET /api/downloads/stats HTTP/1.1" 200 98 | stdout 172.17.0.1 - - [02/Feb/2021 07:04:46] "GET /api/downloads HTTP/1.1" 200 12547 | stdout Added url https://www.youtube.com/watch?v=dFgu6Mb6KKg to the download queue | stdout 172.17.0.1 - - [02/Feb/2021 07:04:50] "POST /api/downloads HTTP/1.1" 200 100 | stdout Error during download task: | stdout AttributeError 'NoneType' object has no attribute 'startswith' | stdout

How to fix it？ Thanks a lot.

[nbr23]

Error during download task: | stdout AttributeError 'NoneType' object has no attribute 'startswith' | stdout

https://github.com/nbr23/youtube-dl-server/commit/91d9ef2e43a13288bd24e93202ba0c558f754f9c should fix the issue you are getting.

Thanks!

[PDSCi996]

Error during download task: | stdout AttributeError 'NoneType' object has no attribute 'startswith' | stdout

91d9ef2 should fix the issue you are getting.

Thanks!

Thank you very much for your hard work. This improvement has solved the problem very well. Now the problem of monitoring YouTube has been perfectly solved with flexget. You can download the daily updates of YouTube happily. In the past few days, I saw a new youtube-dl container, which is very convenient to monitor YouTube updates. The address is "https://github.com/Jeeaaasus/youtube-dl". I suggest you to refer to the integration After all, it is very convenient to monitor and download with rss and other functions. I think the metadata, thumbnails and subtitles of the YouTube video author are very useful. I just use it with jellyfin, so I added the following configuration in "config.yml" "geo-bypass: True abort-on-unavailable-fragment: True mergo-outpu-format: mp4 write-sub: True sub-lang: gb embed-subs: True embed-thumbnail: True add-metadate: True convert-subs: srt" The container can start normally, but the system prompts me an error when downloading "Usage: youtube-dl [OPTIONS] URL [URL...] youtube-dl: error: no such option: --mergo-outpu-format Usage: youtube-dl [OPTIONS] URL [URL...] youtube-dl: error: no such option: --add-metadate" I can start the download after deleting these two parameters, but an error is prompted after completion "[G4CBtbGBy8Q].webp" to JPEG ERROR: AtomicParsley was not found. Please install." Then it was over and the download was unsuccessful. I checked, these are the regular parameters of youtube-dl, I don’t know where I used them wrong? Thanks again.

[nbr23]

For the AtomicParsley error (from --embed-thumbnail) you should use this docker tag: docker pull nbr23/youtube-dl-server:youtube-dl_atomicparsley

The other errors seem to be typos.

• --mergo-outpu-format should be --merge-output-format
• --add-metadate should be --add-metadata

[PDSCi996]

For the AtomicParsley error (from --embed-thumbnail) you should use this docker tag: docker pull nbr23/youtube-dl-server:youtube-dl_atomicparsley

The other errors seem to be typos.

• --mergo-outpu-format should be --merge-output-format
• --add-metadate should be --add-metadata

Sorry, I was careless and added extra trouble to you. I have entered the parameters correctly, "Ydl_server: # youtube-dl-server specific settings port: 8080 # Port youtube-dl-server should listen on host: 0.0.0.0 # IP youtube-dl-server should bind to debug: False # Enable/Disable debug mode metadata_db_path:'/youtube-dl/.ydl-metadata.db' # Path to metadata DB output_playlist:'/youtube-dl/%(playlist)s/%(playlist_index)s-%(title)s.%(ext)s' # Playlist output directory update_poll_delay_min: 1440 # Automatically check for updates every 24h

ydl_options: # youtube-dl options output:'/youtube-dl/%(uploader)s/%(title)s.%(ext)s' # output directory cache-dir:'/youtube-dl/.cache' # youtube-dl cache directory ignore-errors: True # instruct youtube-dl to skip errors age-limit: 6 # minimal age requirement / parental control setting format: bestvideo[ext=mp4][height<=1080]+bestaudio[ext=m4a]/best[ext=mp4]/best geo-bypass: True abort-on-unavailable-fragment: True merge-output-format: mp4 write-sub: True sub-lang: gb embed-subs: True embed-thumbnail: True add-metadata: True convert-subs: srt" Download everything is normal, but the following error is prompted when merging files "ERROR: AtomicParsley was not found. Please install." I checked the information and said that it is necessary to install an additional plug-in software to embed video metadata files. I wonder if the "AtomicParsley" function will be added in subsequent versions.

[nbr23]

For AtomicParsley you need to use the docker image nbr23/youtube-dl-server:youtube-dl_atomicparsley

[PDSCi996]

For AtomicParsley you need to use the docker image nbr23/youtube-dl-server:youtube-dl_atomicparsley

Thank you very much. I have downloaded the docker image "nbr23/youtube-dl-server:youtube-dl_atomicparsley" and made a new deployment, but the image on February 4th seems to have a new bug. I put the parameters on line 11 of "config.yml" output:'/youtube-dl/%(title)s.%(ext)s' # output directory change to output:'/youtube-dl/%(uploader)s/%(title)s.%(ext)s' # output directory When the container cannot be started, the log shows Using configuration file /app_config/config.yml Traceback (most recent call last): File "/usr/src/app/./youtube-dl-server.py", line 14, in from ydl_server.routes import routes File "/usr/src/app/ydl_server/routes.py", line 10, in finished_files = StaticFiles(directory=Path(app_config['ydl_options'].get('output')).parent) File "/usr/local/lib/python3.9/site-packages/starlette/staticfiles.py", line 57, in init raise RuntimeError(f"Directory'{directory}' does not exist") RuntimeError: Directory'/youtube-dl/%(uploader)s' does not exist

I don’t know if it’s a problem with my settings. I added the following parameters in "config.yml" format: bestvideo[ext=mp4][height<=1080]+bestaudio[ext=m4a]/best[ext=mp4]/best geo-bypass: True abort-on-unavailable-fragment: True merge-output-format: mp4 recode-video: mp4 write-thumbnail: True write-sub: True sub-lang: zh-CN,zh-TW,zh-Hans,zh sub-format: srt embed-subs: True embed-thumbnail: True add-metadata: True convert-subs: srt The subtitles and metadata of the video can be added to the video normally, but the thumbnail is only converted from the original "webp" to the "jpg" format. A video file and a thumbnail file are generated in the directory, and the log prompts [ffmpeg] Converting subtitles [ffmpeg] There aren't any subtitles to embed [ffmpeg] Converting thumbnail "/youtube-dl/Farmer’s food, you will have appetite when you smell it, and it tastes great, it’s super!.webp" to JPEG [atomicparsley] Adding thumbnail to "/youtube-dl/Farmer’s food, you will have appetite when you smell it, and it tastes great, super meal!.mp4" I don't know if my parameter settings are wrong or other problems.

[nbr23]

Oh, interesting find this one. Thanks, I am working on a fix.

[nbr23]

@PDSCi996 https://github.com/nbr23/youtube-dl-server/commit/566ebe3ea9f1297bf2367e3aa3564ea991751bbb should fix your issue. Thanks, let me know if that works!

[PDSCi996]

@PDSCi996 566ebe3 should fix your issue. Thanks, let me know if that works!

Thank you for your hard work. I use "youtube-dl_atomicparsley" in dockerhub This new container is deployed, the config.yml parameters are as follows ydl_server: # youtube-dl-server specific settings port: 8080 # Port youtube-dl-server should listen on host: 0.0.0.0 # IP youtube-dl-server should bind to debug: False # Enable/Disable debug mode metadata_db_path:'/youtube-dl/.ydl-metadata.db' # Path to metadata DB output_playlist:'/youtube-dl/%(playlist)s/%(playlist_index)s-%(title)s.%(ext)s' # Playlist output directory template update_poll_delay_min: 1440 # Automatically check for updates every 24h max_log_entries: 100 # Maximum number of job log history to keep forwarded_allow_ips: None # uvicorn Comma seperated list of IPs to trust with proxy headers. proxy_headers: True # uvicorn flag Enable/Disable X-Forwarded-Proto, X-Forwarded-For, X-Forwarded-Port to populate remote address info.

ydl_options: # youtube-dl options output:'/youtube-dl/%(uploader)s/%(title)s.%(ext)s' # output directory template cache-dir:'/youtube-dl/.cache' # youtube-dl cache directory ignore-errors: True # instruct youtube-dl to skip errors age-limit: 6 # minimal age requirement / parental control setting format: bestvideo[ext=mp4][height<=1080]+bestaudio[ext=m4a]/best[ext=mp4]/best geo-bypass: True abort-on-unavailable-fragment: True merge-output-format: mp4 recode-video: mp4 write-thumbnail: True write-sub: True sub-lang: zh-CN,zh-TW,zh-Hans,zh

all-subs: True

sub-format: srt embed-subs: True embed-thumbnail: True add-metadata: True convert-subs: srt The startup is normal, and the "output:'/youtube-dl/%(uploader)s/%(title)s.%(ext)s'" parameter has been able to function normally, but "AtomicParsley" seems to be back to the old way, I Try to download a video, the log is as follows [youtube] n6lE6qMKUdU: Downloading webpage [youtube] n6lE6qMKUdU: Downloading thumbnail ... [youtube] n6lE6qMKUdU: Writing thumbnail to: /youtube-dl/WarLeaks-Military Blog/Marines Amphibious Assault Vehicles Launch From Amphibious Transport Dock.webp [download] Destination: /youtube-dl/WarLeaks-Military Blog/Marines Amphibious Assault Vehicles Launch From Amphibious Transport Dock.f137.mp4 [download] 100% of 61.06MiB in 00:52 [download] Destination: /youtube-dl/WarLeaks-Military Blog/Marines Amphibious Assault Vehicles Launch From Amphibious Transport Dock.f140.m4a [download] 100% of 1.98MiB in 00:05 [ffmpeg] Merging formats into "/youtube-dl/WarLeaks-Military Blog/Marines Amphibious Assault Vehicles Launch From Amphibious Transport Dock.mp4" Deleting original file /youtube-dl/WarLeaks-Military Blog/Marines Amphibious Assault Vehicles Launch From Amphibious Transport Dock.f137.mp4 (pass -k to keep) Deleting original file /youtube-dl/WarLeaks-Military Blog/Marines Amphibious Assault Vehicles Launch From Amphibious Transport Dock.f140.m4a (pass -k to keep) [ffmpeg] Not converting video file /youtube-dl/WarLeaks-Military Blog/Marines Amphibious Assault Vehicles Launch From Amphibious Transport Dock.mp4-already is in target format mp4 [ffmpeg] Adding metadata to'/youtube-dl/WarLeaks-Military Blog/Marines Amphibious Assault Vehicles Launch From Amphibious Transport Dock.mp4' [ffmpeg] There aren't any subtitles to convert [ffmpeg] There aren't any subtitles to embed [ffmpeg] Converting thumbnail "/youtube-dl/WarLeaks-Military Blog/Marines Amphibious Assault Vehicles Launch From Amphibious Transport Dock.webp" to JPEG ERROR: AtomicParsley was not found. Please install. Hope that helps. youtube-dlc-AtomicParsley was also tested, the same problem.

[nbr23]

Ok, it turns out alpine installs AtomicParsley as atomicparsley and youtube-dl looks for a binary called AtomicParsley so youtube-dl was not finding it, even when installed. I have a workaround in place here: https://github.com/nbr23/youtube-dl-server/commit/413182c68d97792abb69455e8739c85dce27ef0d

I also submitted a PR to youtube-dl in case it can be fixed there https://github.com/ytdl-org/youtube-dl/pull/28112

We will get there :)

[PDSCi996]

Ok, it turns out alpine installs AtomicParsley as atomicparsley and youtube-dl looks for a binary called AtomicParsley so youtube-dl was not finding it, even when installed. I have a workaround in place here: 413182c

I also submitted a PR to youtube-dl in case it can be fixed there ytdl-org/youtube-dl#28112

We will get there :)

Seeing your news, I came right away. I have re-deployed the latest "youtube-dl_atomicparsley "Mirror, log shows normal, as shown below [youtube] QmMvEQJQDN8: Downloading webpage WARNING: zh-CN subtitles not available for QmMvEQJQDN8 WARNING: zh-TW subtitles not available for QmMvEQJQDN8 WARNING: zh-Hans subtitles not available for QmMvEQJQDN8 WARNING: zh subtitles not available for QmMvEQJQDN8 [youtube] QmMvEQJQDN8: Downloading thumbnail ... [youtube] QmMvEQJQDN8: Writing thumbnail to: /youtube-dl/Home Cooking Diary/Why is the restaurant’s mixed cucumber so delicious? It turned out to be so simple, 5 pieces at a time are not enough to eat Cold cucumber.webp [download] Destination: /youtube-dl/Home Cooking Diary/Why is the restaurant's mixed cucumber so delicious? It turned out to be so simple, 5 pieces at a time are not enough to eat Cold cucumber.f299.mp4 [download] 100% of 108.27MiB in 01:28 [download] Destination: /youtube-dl/Home Cooking Diary/Why is the restaurant's mixed cucumber so delicious? It turned out to be so simple, 5 pieces at a time are not enough to eat Cold cucumber.f140.m4a [download] 100% of 2.69MiB in 00:03 [ffmpeg] Merging formats into "/youtube-dl/Home Cooking Diary/Why is the restaurant's mixed cucumber so delicious? It turned out to be so simple, 5 pieces at a time are not enough to eat Cold cucumber.mp4" Deleting original file /youtube-dl/Home Cooking Diary/Why is the restaurant's mixed cucumber so delicious? It turned out to be as simple as 5 pieces at a time. Cold cucumber.f299.mp4 (pass -k to keep) Deleting original file /youtube-dl/Home Cooking Diary/Why is the restaurant's mixed cucumber so delicious? It turned out to be as simple as 5 pieces at a time. Cold cucumber.f140.m4a (pass -k to keep) [ffmpeg] Adding metadata to'/youtube-dl/Home Cooking Diary/Why is the restaurant's mixed cucumber so delicious? It turned out to be so simple, 5 pieces at a time are not enough to eat Cold cucumber.mp4' [ffmpeg] Converting subtitles [ffmpeg] There aren't any subtitles to embed [ffmpeg] Converting thumbnail "/youtube-dl/Home Cooking Diary/Why is the restaurant's mixed cucumber so delicious? It turned out to be so simple, 5 pieces at a time are not enough to eat Cold cucumber.webp" to JPEG [atomicparsley] Adding thumbnail to "/youtube-dl/Home Cooking Diary/Why is the restaurant's mixed cucumber so delicious? It turned out to be so simple, 5 pieces at a time are not enough to eat Cold cucumber.mp4"

However, for the two files generated in the hard disk directory, an mp4 and a jpg file, although the log display is correct, the picture file cannot be merged into the video file.

[nbr23]

I think it does actually work. If you can get a shell on your docker container and run atomicparsley yourfile.mp4 -E, you will see that it does extract a jpeg thumbnail from the video.

The reason why you still have the jpg file written on your disk is the write-thumbnail: True parameter. Remove it and only keep the embed-thumbnail and you should be fine.

[PDSCi996]

I think it does actually work. If you can get a shell on your docker container and run atomicparsley yourfile.mp4 -E, you will see that it does extract a jpeg thumbnail from the video.

The reason why you still have the jpg file written on your disk is the write-thumbnail: True parameter. Remove it and only keep the embed-thumbnail and you should be fine.

As you said, I adjusted the parameter settings, and the thumbnails can be merged into the video file normally. It seems that my understanding of youtube-dl is not deep enough, and I will learn more in the future. Thank you again for your hard work over many days.

[PDSCi996]

Sorry, there is a new bug.

The first bug is the prompt that appears when the youtube-dl is automatically updated when the container is restarted. The log is as follows. It appeared several times, and then it did not appear. Requirement already satisfied: youtube-dl in /usr/local/lib/python3.9/site-packages (2021.2.4.1) WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by'ReadTimeoutError("HTTPSConnectionPool(host='pypi.org', port=443): Read timed out. (read timeout=15)")': /simple/youtube-dl/ WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by'ReadTimeoutError("HTTPSConnectionPool(host='pypi.org', port=443): Read timed out. (read timeout=15)")': /simple/youtube-dl/

The second bug is that I manually filled in a playlist to download. It prompted that only the first video can be downloaded, and the rest of the videos will not be downloaded. The log is as follows. This problem also occurred several times, and then it did not recur. [youtube:tab] Downloading playlist PLA1VDIqub4k_LDR_G6_T_fYm848yK8i_q-add --no-playlist to just download video st_AakadXdk [youtube:tab] PLA1VDIqub4k_LDR_G6_T_fYm848yK8i_q: Downloading webpage [youtube:tab] PLA1VDIqub4k_LDR_G6_T_fYm848yK8i_q: Downloading webpage

The third bug is that if you download a playlist, if a directory named "/youtube-dl" has been created under the directory set by "/youtube-dl", a new directory named "channel name-videos" will be created at this time. Will not put the videos of the same channel in one directory.

[nbr23]

1 - I am not able to reproduce. From the error you are getting, it seems it had issues connecting to pypi.org at the time, and as such the upgrade failed. This is most likely either a local issue on your end or a problem with pypi.org at the time.

2 - The logs are telling you that it is downloading the whole playlist and that IF you only want the one video, you can add --no-playlist. As you have not added this flag, you should be getting the full playlist. Please double check in your download directory and confirm.

3 - Can you share your current config and give me specific examples of what you are downloading, where it ends up, and where you would want it to end up? (by the way, when you paste your config, you can use ``` before and after the code so that github doesn't break the format ;) )

[PDSCi996]

1 - I am not able to reproduce. From the error you are getting, it seems it had issues connecting to pypi.org at the time, and as such the upgrade failed. This is most likely either a local issue on your end or a problem with pypi.org at the time.

2 - The logs are telling you that it is downloading the whole playlist and that IF you only want the one video, you can add --no-playlist. As you have not added this flag, you should be getting the full playlist. Please double check in your download directory and confirm.

3 - Can you share your current config and give me specific examples of what you are downloading, where it ends up, and where you would want it to end up? (by the way, when you paste your config, you can use ``` before and after the code so that github doesn't break the format ;) )

I will pay attention to adding "" in the future. This is the complete configuration of my "config.yml"

“ydl_server: # youtube-dl-server specific settings port: 8080 # Port youtube-dl-server should listen on host: 0.0.0.0 # IP youtube-dl-server should bind to debug: False # Enable/Disable debug mode metadata_db_path: '/youtube-dl/.ydl-metadata.db' # Path to metadata DB output_playlist: '/youtube-dl/%(playlist)s/%(playlist_index)s - %(title)s.%(ext)s' # Playlist output directory template update_poll_delay_min: 1440 # Automatically check for updates every 24h max_log_entries: 100 # Maximum number of job log history to keep forwarded_allow_ips: None # uvicorn Comma seperated list of IPs to trust with proxy headers. proxy_headers: True # uvicorn flag Enable/Disable X-Forwarded-Proto, X-Forwarded-For, X-Forwarded-Port to populate remote address info.

ydl_options: # youtube-dl options output: '/youtube-dl/%(uploader)s/%(title)s.%(ext)s' # output directory template cache-dir: '/youtube-dl/.cache' # youtube-dl cache directory ignore-errors: True # instruct youtube-dl to skip errors age-limit: 6 # minimal age requirement / parental control setting format: bestvideo[ext=mp4][height<=1080]+bestaudio[ext=m4a]/best[ext=mp4]/best geo-bypass: True abort-on-unavailable-fragment: True

playlist-end: 20

match-filter: "! is_live" playlist-reverse: True merge-output-format: mp4 recode-video: mp4

write-thumbnail: True

write-sub: True sub-lang: zh-CN,zh-TW,zh-Hans,zh

all-subs: True

sub-format: srt embed-subs: True embed-thumbnail: True add-metadata: True convert-subs: srt”

This address is where I want to download the playlist. “https://www.youtube.com/channel/UCUXi5mmqbvIithAs9AaxEtw/videos”

[nbr23]

It's actually three backquotes ` not regular quotes to format as code ;) You can also select your code and use this button in github

Thanks for the info, I am checking your issue.

[nbr23]

That config works fine for me and downloads the full playlist in one folder under /youtube-dl named with the title of the playlist:

[PDSCi996]

That config works fine for me and downloads the full playlist in one folder under /youtube-dl named with the title of the playlist:

I will first use flexget to automatically push the YouTube channels I have followed for a long time to the youtube-dl container every day. The videos downloaded by each different channel will be saved in a directory named after the channel name, and I will also download some A previous video of a channel that has been following for a long time. At this time, a "channel name + videos" directory will be automatically created. The channel directory created by flexget automatic push is two different directories. At this time, you must manually put the two directories under Videos are merged into one directory, and your screenshots show that "videos" is added to the end of the directory. This "videos" is not the channel name. Do not know what is the reasons of that. Thank you for your prompt reply.

[PDSCi996]

It's actually three backquotes ` not regular quotes to format as code ;) You can also select your code and use this button in github

Thanks for the info, I am checking your issue.

Thanks, I'm still new to GitHub.

[nbr23]

That config works fine for me and downloads the full playlist in one folder under /youtube-dl named with the title of the playlist:

I will first use flexget to automatically push the YouTube channels I have followed for a long time to the youtube-dl container every day. The videos downloaded by each different channel will be saved in a directory named after the channel name, and I will also download some A previous video of a channel that has been following for a long time. At this time, a "channel name + videos" directory will be automatically created. The channel directory created by flexget automatic push is two different directories. At this time, you must manually put the two directories under Videos are merged into one directory, and your screenshots show that "videos" is added to the end of the directory. This "videos" is not the channel name. Do not know what is the reasons of that. Thank you for your prompt reply.

Ultimately I think this is more of a youtube-dl issue / trick than something link to youtube-dl-server. The options you are setting are being fed and understood properly by youtube-dl. If the behavior of youtube-dl is not what you expect, this is not something I can act on in youtube-dl-server and you will need to check youtube-dl's doc / issues.