search

mikenye / docker-youtube-dl

Youtube-DL container with all prerequisites (libav-tools, ffmpeg, rtmpdump, mplayer, mpv)
225 stars 42 forks source link

Docker-compose needed for subscriptions #16

Closed ghost closed 3 years ago

ghost commented 3 years ago

Thanks for great work.

Can you please include also docker-compose for subscription with netrc file. Thanks for your help.

mikenye commented 3 years ago

Hi @krswin,

You want an example using docker-compose to download subscriptions?

ghost commented 3 years ago

Yes. correct subscriptions after login to my youtube account. Thanks

mikenye commented 3 years ago

Does this work for you?

version: '3.8'

services:
  youtube-dl:
    image: mikenye/youtube-dl:latest
    tty: true
    container_name: youtube-dl-cron
environment:
      - "PUID=YOURUID"
      - "PGID=YOURGID"
    volumes:
      - /path/to/downloads:/workdir:rw
    command:
      - :ytsubscriptions
      - --dateafter
      -   now-5days
      - --download-archive
      -   /workdir/.youtube-dl-archive
      - --cookies
      -   /workdir/.youtube-dl-cookiejar
      - --limit-rate
      -   "5000"
      - -v
      - --ignore-errors
      - --no-overwrites
      - --continue
      - --no-post-overwrites
      - --add-metadata
      - --write-thumbnail
      - --playlist-reverse
      - --write-description
      - --write-info-json
      - --write-annotations
      - --format
      -   "best[height<=?1080]"
      - --fixup
      -   fix
      - --download-archive
      -   '/workdir/.download_archive'
      - --output
      -   '/workdir/%(uploader)s/%(uploader)s - %(upload_date)s - %(title)s - %(id)s.%(ext)s'
      - --username
      -   YOURUSERNAME
      - --password
      -   YOURPASSWORD
ghost commented 3 years ago

@mikenye Thanks for your kind help. I am trying it, but how can I pass the youtube video URL to download? or youtube channel URL to download? Thanks.

mikenye commented 3 years ago

Hi @krswin,

In the YAML above, see the line:

    - :ytsubscriptions

You can replace that with the URL of your choice, eg:

    - https://youtu.be/dQw4w9WgXcQ
ghost commented 3 years ago

Hi @mikenye Thanks for reply. I have tried but I see one error : youtube-dl-cron | [youtube:tab] Downloading login page youtube-dl-cron | WARNING: unable to fetch login page: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:727)> youtube-dl-cron | [youtube:tab] UCuAXFkgsw1L7xaCfnd5JJOw: Downloading webpage youtube-dl-cron | ERROR: Unable to download webpage: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:727)> (caused by URLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:727)'),)) youtube-dl-cron | File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 632, in _request_webpage youtube-dl-cron | return self._downloader.urlopen(url_or_request) youtube-dl-cron | File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2248, in urlopen youtube-dl-cron | return self._opener.open(req, timeout=self._socket_timeout) youtube-dl-cron | File "/usr/lib/python2.7/urllib2.py", line 429, in open youtube-dl-cron | response = self._open(req, data) youtube-dl-cron | File "/usr/lib/python2.7/urllib2.py", line 447, in _open youtube-dl-cron | '_open', req) youtube-dl-cron | File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain youtube-dl-cron | result = func(*args) youtube-dl-cron | File "/usr/local/bin/youtube-dl/youtube_dl/utils.py", line 2736, in https_open youtube-dl-cron | req, **kwargs) youtube-dl-cron | File "/usr/lib/python2.7/urllib2.py", line 1198, in do_open youtube-dl-cron | raise URLError(err) youtube-dl-cron | youtube-dl-cron exited with code 1

mikenye commented 3 years ago

Can you post your docker-compose.yml file?

If you have a user/pass be sure to remove before posting.

ghost commented 3 years ago

Thanks for reply. Its the same like you sent. 

version: '3.8'

services:
  youtube-dl:
    image: mikenye/youtube-dl:latest
    tty: true
    container_name: youtube-dl-cron
    environment:
      - "PUID=1001"
      - "PGID=1001"
    volumes:
      - ./downloads:/workdir:rw
    command:
      - https://www.youtube.com/channel/UCuAXFkgsw1L7xaCfnd5JJOw
      - --dateafter
      -   now-5days
      - --download-archive
      -   /workdir/.youtube-dl-archive
      - --cookies
      -   /workdir/.youtube-dl-cookiejar
      - --limit-rate
      -   "5000"
      - -v
      - --ignore-errors
      - --no-overwrites
      - --continue
      - --no-post-overwrites
      - --add-metadata
      - --write-thumbnail
      - --playlist-reverse
      - --write-description
      - --write-info-json
      - --write-annotations
      - --format
      -   "best[height<=?1080]"
      - --fixup
      -   fix
      - --download-archive
      -   '/workdir/.download_archive'
      - --output
      -   '/workdir/%(uploader)s/%(uploader)s - %(upload_date)s - %(title)s - %(id)s.%(ext)s'
      - --username
      -   YOURUSERNAME
      - --password
      -   YOURPASSWORD
mikenye commented 3 years ago

Hi @krswin,

You might need to remove:

      - --dateafter
      -   now-5days

... as the videos in the playlist in your compose are older than 5 days old.

Secondly, the certificate error makes me think you could be running an older version of the image. Try issuing the command docker-compose pull to update the image, then retry your docker-compose up.

ghost commented 3 years ago

@mikenye Happy new year! Thanks for reply. I dig a bit and found the issue, i am running on raspberry pi and your image somehow runs but have issue with SSL certificate, so I have built myself and now it works fine. thanks a lot.

mikenye commented 3 years ago

Thanks for letting me know. I'll look into the SSL issue...

mikenye commented 3 years ago

Happy new year!

ghost commented 3 years ago

So basic idea is that I run docker-compose up -d with cron job every day? to check and download new videos each time its available ?

mikenye commented 3 years ago

Basically yes.

Given you have to build the image yourself, I'd probably do the following.

1. Clone the mikenye/docker-youtube-dl repo, eg:

git clone https://github.com/mikenye/docker-youtube-dl.git /opt/youtube-dl/src

2. Create your compose file in /opt/youtube-dl

Remove the the image: directive and add a build: section as follows:

version: '3.8'

services:
  youtube-dl:
    build:
      context: ./src
    tty: true
    container_name: youtube-dl-cron
    environment:
      - "PUID=1001"
      - "PGID=1001"
    volumes:
      - ./downloads:/workdir:rw
    command:
      - https://www.youtube.com/channel/UCuAXFkgsw1L7xaCfnd5JJOw
      - --dateafter
      -   now-5days
      - --download-archive
      -   /workdir/.youtube-dl-archive
      - --cookies
      -   /workdir/.youtube-dl-cookiejar
      - --limit-rate
      -   "5000"
      - -v
      - --ignore-errors
      - --no-overwrites
      - --continue
      - --no-post-overwrites
      - --add-metadata
      - --write-thumbnail
      - --playlist-reverse
      - --write-description
      - --write-info-json
      - --write-annotations
      - --format
      -   "best[height<=?1080]"
      - --fixup
      -   fix
      - --download-archive
      -   '/workdir/.download_archive'
      - --output
      -   '/workdir/%(uploader)s/%(uploader)s - %(upload_date)s - %(title)s - %(id)s.%(ext)s'
      - --username
      -   YOURUSERNAME
      - --password
      -   YOURPASSWORD

3. Set up cron to regularly (weekly) build the image.

Add the following to your crontab:

# Ensure youtube-dl container source is up to date
00 1    * * 7   root  cd /opt/youtube-dl/src && git pull
# Build the image
30 1    * * 7   root  cd /opt/youtube-dl && docker-compose build --no-cache --pull
# Remove unused images
00 2    * * 7   root  docker image prune --all -f

This will refresh the source code and rebuild the image every Sunday early in the morning.

4. Set up cron to download you videos.

Add the following to your crontab:

# Download vids
30 2    * * *   root  cd /opt/youtube-dl && docker-compose up --remove-orphans

This will download videos daily at 2:30am.

ghost commented 3 years ago

Great Thanks for your help.

ghost commented 3 years ago

@mikenye I was checking and All the things works fine, except Login to youtube. Did it work for you? my error is youtube-dl-cron | WARNING: Unable to look up account info: HTTP Error 400: Bad Request.

I have already tried with the docker-compose username password settings and also I have just tried by
- --netrc and mapping to ./config/.netrc:/home/dockeruser/.netrc:ro

but still the same error. Thanks.

mikenye commented 3 years ago

Hi @krswin unfortunately this is a known issue.

Try following these steps to fix:

https://github.com/ytdl-org/youtube-dl/issues/21313#issuecomment-499496235

ghost commented 3 years ago

Thanks for reply. Yes I tried everything to make this login work but not working. but thanks it seems its ytdl issue.