search

ytdl-org / youtube-dl

Command-line program to download videos from YouTube.com and other video sites
http://ytdl-org.github.io/youtube-dl/
The Unlicense
132k stars 10.01k forks source link

Erocast #31203

Open FranekKemono opened 2 years ago

FranekKemono commented 2 years ago

Example URLs

https://erocast.me/track/4254/intimate-morning-with-your-wife

Description

Popular website for adult voice actors from /r/gonewildaudio

dirkf commented 2 years ago

The audio stream and metadata are in JSON assigned to var song_data_{audio_id} (here audo_id == '4254'), like this:

pretty-printed JS from page 
var song_data_4254 = {
  'id': 4254,
  'mp3': 0,
  'waveform': 0,
  'preview': 0,
  'wav': 0,
  'flac': 0,
  'hd': 0,
  'hls': 1,
  'explicit': 0,
  'selling': 1,
  'price': '0.00',
  'genre': '10,6',
  'mood': null,
  'title': 'Intimate morning  with your wife',
  'description': null,
  'access': null,
  'duration': 1409,
  'artistIds': null,
  'loves': 0,
  'collectors': 0,
  'plays': 2344,
  'released_at': '08/26/2022',
  'copyright': null,
  'allow_download': 0,
  'download_count': 0,
  'allow_comments': 1,
  'comment_count': 0,
  'visibility': 1,
  'approved': 1,
  'pending': 0,
  'created_at': '2022-08-26T19:20:14.000000Z',
  'updated_at': '2022-08-27T11:49:34.000000Z',
  'vocal': 1,
  'script': null,
  'artwork_url': 'https://erocast.me/common/default/song.png',
  'permalink_url': 'https://erocast.me/track/4254/intimate-morning-with-your-wife',
  'stream_url': 'https://erocast.me/stream/hls/4254',
  'favorite': false,
  'library': false,
  'streamable': true,
  'user': {
    'id': 4198,
    'name': 'ZLOY_ASMR',
    'username': 'ZLOY_ASMR',
    'session_id': null,
    'artist_id': 0,
    'collection_count': 0,
    'following_count': 0,
    'follower_count': 0,
    'last_activity': '2022-08-26 19:41:03',
    'notification': null,
    'bio': null,
    'allow_comments': 1,
    'comment_count': 0,
    'script': null,
    'artwork_url': 'https://erocast.me/common/default/user.png',
    'permalink_url': 'https://erocast.me/zloy-asmr'
  },
  'tags': [
    {
      'id': 48044,
      'song_id': 4254,
      'tag': 'morning',
      'permalink_url': 'https://erocast.me/tracks/tag/morning'
    },
    {
      'id': 48045,
      'song_id': 4254,
      'tag': 'established relationships',
      'permalink_url': 'https://erocast.me/tracks/tag/established-relationships'
    },
    {
      'id': 48046,
      'song_id': 4254,
      'tag': 'wholesome',
      'permalink_url': 'https://erocast.me/tracks/tag/wholesome'
    },
    {
      'id': 48047,
      'song_id': 4254,
      'tag': 'kisses',
      'permalink_url': 'https://erocast.me/tracks/tag/kisses'
    },
    {
      'id': 48048,
      'song_id': 4254,
      'tag': 'wife',
      'permalink_url': 'https://erocast.me/tracks/tag/wife'
    }
  ]
}
abhishek-1309 commented 2 years ago

Can I take this?

dirkf commented 2 years ago

Thanks.

abhishek-1309 commented 2 years ago

I wrote following class as per the guidlines of adding support to new site 

class ErocastIE(InfoExtractor):
    _VALID_URL = r'https?://(?:www\.)?erocast\.me/track/(?P<id>[0-9]+)/([a-z]+-)+[a-z]+'
    _TEST = {
        'url': 'https://erocast.me/track/4254/intimate-morning-with-your-wife',
        'md5': 'TODO: md5 sum of the first 10241 bytes of the video file (use --test)',
        'info_dict': {
            'id': 4254,
            'ext': 'mp3',
            'mp3': 0,
            'waveform': 0,
            'preview': 0,
            'wav': 0,
            'flac': 0,
            'hd': 0,
            'hls': 1,
            'explicit': 0,
            'selling': 1,
            'price': '0.00',
            'genre': '10,6',
            'mood': None,
            'title': 'Intimate morning  with your wife',
            'description': None,
            'access': None,
            'duration': 1409,
            'artistIds': None,
            'loves': 0,
            'collectors': 0,
            'plays': 2344,
            'released_at': '08/26/2022',
            'copyright': None,
            'allow_download': 0,
            'download_count': 0,
            'allow_comments': 1,
            'comment_count': 0,
            'visibility': 1,
            'approved': 1,
            'pending': 0,
            'created_at': '2022-08-26T19:20:14.000000Z',
            'updated_at': '2022-08-27T11:49:34.000000Z',
            'vocal': 1,
            'script': None,
            'artwork_url': 'https://erocast.me/common/default/song.png',
            'permalink_url': 'https://erocast.me/track/4254/intimate-morning-with-your-wife',
            'stream_url': 'https://erocast.me/stream/hls/4254',
            'favorite': False,
            'library': False,
            'streamable': True,
            'user': {
                'id': 4198,
                'name': 'ZLOY_ASMR',
                'username': 'ZLOY_ASMR',
                'session_id': None,
                'artist_id': 0,
                'collection_count': 0,
                'following_count': 0,
                'follower_count': 0,
                'last_activity': '2022-08-26 19:41:03',
                'notification': None,
                'bio': None,
                'allow_comments': 1,
                'comment_count': 0,
                'script': None,
                'artwork_url': 'https://erocast.me/common/default/user.png',
                'permalink_url': 'https://erocast.me/zloy-asmr'
            },
            'tags': [
                {
                    'id': 48044,
                    'song_id': 4254,
                    'tag': 'morning',
                    'permalink_url': 'https://erocast.me/tracks/tag/morning'
                },
                {
                    'id': 48045,
                    'song_id': 4254,
                    'tag': 'established relationships',
                    'permalink_url': 'https://erocast.me/tracks/tag/established-relationships'
                },
                {
                    'id': 48046,
                    'song_id': 4254,
                    'tag': 'wholesome',
                    'permalink_url': 'https://erocast.me/tracks/tag/wholesome'
                },
                {
                    'id': 48047,
                    'song_id': 4254,
                    'tag': 'kisses',
                    'permalink_url': 'https://erocast.me/tracks/tag/kisses'
                },
                {
                    'id': 48048,
                    'song_id': 4254,
                    'tag': 'wife',
                    'permalink_url': 'https://erocast.me/tracks/tag/wife'
                }
            ]
            # TODO more properties, either as:
            # * A value
            # * MD5 checksum; start the string with md5:
            # * A regular expression; start the string with re:
            # * Any Python type (for example int or float)
        }
    }

While running the test, I am getting this error, am I missing anything?

PS C:\Users\Abhishek\youtube-dl> python test/test_download.py TestDownload.test_Erocast
[Erocast] 4254: Downloading webpage
ERROR: unable to extract uploader; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and
include its complete output.; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and inclu
de its complete output.
Traceback (most recent call last):
  File "C:\Users\Abhishek\youtube-dl\youtube_dl\YoutubeDL.py", line 816, in wrapper
    return func(self, *args, **kwargs)
  File "C:\Users\Abhishek\youtube-dl\youtube_dl\YoutubeDL.py", line 837, in __extract_info
    ie_result = ie.extract(url)
  File "C:\Users\Abhishek\youtube-dl\youtube_dl\extractor\common.py", line 534, in extract
    ie_result = self._real_extract(url)
  File "C:\Users\Abhishek\youtube-dl\youtube_dl\extractor\erocast.py", line 123, in _real_extract
    'uploader': self._search_regex(r'<div[^>]+id="uploader"[^>]*>([^<]+)<', webpage, 'uploader', fatal=False),
  File "C:\Users\Abhishek\youtube-dl\youtube_dl\extractor\common.py", line 1014, in _search_regex
    self._downloader.report_warning('unable to extract %s' % _name + bug_reports_message())
  File "C:\Users\Abhishek\youtube-dl\test\helper.py", line 271, in _report_warning
    real_warning(w)
  File "C:\Users\Abhishek\youtube-dl\test\test_download.py", line 53, in report_warning
    raise ExtractorError(message)
youtube_dl.utils.ExtractorError: unable to extract uploader; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl wi
th the --verbose flag and include its complete output.; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with th
e --verbose flag and include its complete output.

E
======================================================================
ERROR: test_Erocast (__main__.TestDownload):
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\Abhishek\youtube-dl\youtube_dl\YoutubeDL.py", line 816, in wrapper
    return func(self, *args, **kwargs)
  File "C:\Users\Abhishek\youtube-dl\youtube_dl\YoutubeDL.py", line 837, in __extract_info
    ie_result = ie.extract(url)
  File "C:\Users\Abhishek\youtube-dl\youtube_dl\extractor\common.py", line 534, in extract
    ie_result = self._real_extract(url)
  File "C:\Users\Abhishek\youtube-dl\youtube_dl\extractor\erocast.py", line 123, in _real_extract
    'uploader': self._search_regex(r'<div[^>]+id="uploader"[^>]*>([^<]+)<', webpage, 'uploader', fatal=False),
  File "C:\Users\Abhishek\youtube-dl\youtube_dl\extractor\common.py", line 1014, in _search_regex
    self._downloader.report_warning('unable to extract %s' % _name + bug_reports_message())
  File "C:\Users\Abhishek\youtube-dl\test\helper.py", line 271, in _report_warning
    real_warning(w)
  File "C:\Users\Abhishek\youtube-dl\test\test_download.py", line 53, in report_warning
    raise ExtractorError(message)
youtube_dl.utils.ExtractorError: unable to extract uploader; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl wi
th the --verbose flag and include its complete output.; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with th
e --verbose flag and include its complete output.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Abhishek\youtube-dl\test\test_download.py", line 167, in test_template
    raise err
  File "C:\Users\Abhishek\youtube-dl\test\test_download.py", line 159, in test_template
    res_dict = ydl.extract_info(
  File "C:\Users\Abhishek\youtube-dl\youtube_dl\YoutubeDL.py", line 809, in extract_info
    return self.__extract_info(url, ie, download, extra_info, process)
  File "C:\Users\Abhishek\youtube-dl\youtube_dl\YoutubeDL.py", line 825, in wrapper
    self.report_error(compat_str(e), e.format_traceback())
  File "C:\Users\Abhishek\youtube-dl\youtube_dl\YoutubeDL.py", line 629, in report_error
    self.trouble(error_message, tb)
  File "C:\Users\Abhishek\youtube-dl\youtube_dl\YoutubeDL.py", line 599, in trouble
    raise DownloadError(message, exc_info)
youtube_dl.utils.DownloadError: ERROR: unable to extract uploader; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube
-dl with the --verbose flag and include its complete output.; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl w
ith the --verbose flag and include its complete output.

----------------------------------------------------------------------
Ran 1 test in 2.217s

FAILED (errors=1)
dirkf commented 2 years ago

You've pasted the test part but not the _real_extract() method.

Please make a draft Pull Request. You'll need to have forked the repo in GitHub, committed your changes to a new branch, and pushed the branch to your fork.

abhishek-1309 commented 2 years ago

I have created a draft pull request https://github.com/ytdl-org/youtube-dl/pull/31261

abhishek-1309 commented 2 years ago

Looks like I have to update the real extract code, just one question how do I know which attributes to return for _real_extract

K-web04 commented 1 year ago

Hey I want to work on this project

dirkf commented 1 year ago

It looks like @abhishek-1309 has given up, but feel free to take it up either way. The suggestions above apply equally.

RonniCGN commented 1 year ago

Hi, I created an extractor for erocast and a pull request https://github.com/ytdl-org/youtube-dl/pull/31631 Only basic functionality to download a single sound file. Erocast uses an m3u8 streaming file. The sound data seems to be in mp4 format (audio only)