search

ytdl-org / youtube-dl

Command-line program to download videos from YouTube.com and other video sites
http://ytdl-org.github.io/youtube-dl/
The Unlicense
131.23k stars 9.93k forks source link

Broken site support: https://www.radiofrance.fr/ to download playlists #31464

Open kdf8jsa1 opened 1 year ago

kdf8jsa1 commented 1 year ago

Checklist

Verbose log

$ youtube-dl -v https://www.radiofrance.fr/personnes/michel-foucault
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://www.radiofrance.fr/personnes/michel-foucault']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Python version 3.8.10 (CPython) - Linux-5.4.0-131-generic-x86_64-with-glibc2.29
[debug] exe versions: ffmpeg 4.2.7, ffprobe 4.2.7, rtmpdump 2.4
[debug] Proxy map: {}
[generic] michel-foucault: Requesting header
WARNING: Falling back on generic information extractor.
[generic] michel-foucault: Downloading webpage
[generic] michel-foucault: Extracting information
ERROR: Unsupported URL: https://www.radiofrance.fr/personnes/michel-foucault
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/youtube_dl/YoutubeDL.py", line 815, in wrapper
    return func(self, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/youtube_dl/YoutubeDL.py", line 836, in __extract_info
    ie_result = ie.extract(url)
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/common.py", line 534, in extract
    ie_result = self._real_extract(url)
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/generic.py", line 3489, in _real_extract
    raise UnsupportedError(url)
youtube_dl.utils.UnsupportedError: Unsupported URL: https://www.radiofrance.fr/personnes/michel-foucault

Description

"radiofrance" is listed in https://ytdl-org.github.io/youtube-dl/supportedsites.html, however all $ youtube-dl https://www.radiofrance.fr/... requests output: ERROR: Unsupported URL.

Please let me know if you need more information from my end. Thank you.

dirkf commented 1 year ago

Try PR #31435, which should resolve this.

kdf8jsa1 commented 1 year ago

Thank you for the info.

I still get an error.

Below is what I did step-by-step, and the error message.

I went to #31435 and copied the two updated files from the PR, locally:

$ wget https://raw.githubusercontent.com/ytdl-org/youtube-dl/72db2172897a76665414fc9da3fc79f096df9fab/youtube_dl/extractor/extractors.py
$ wget https://raw.githubusercontent.com/ytdl-org/youtube-dl/72db2172897a76665414fc9da3fc79f096df9fab/youtube_dl/extractor/radiofrance.py

and removed the 2 files that were deleted in the PR: 

$ sudo rm -i /usr/lib/python3/dist-packages/youtube_dl/extractor/franceculture.py 
$ sudo rm -i /usr/lib/python3/dist-packages/youtube_dl/extractor/franceinter.py

Then replaced updated files locally:

sudo mv radiofrance.py /usr/lib/python3/dist-packages/youtube_dl/extractor/radiofrance.py
sudo mv extractors.py /usr/lib/python3/dist-packages/youtube_dl/extractor/extractors.py

Reboot my laptop.

However, when I run youtube-dl I get an error:

$ youtube-dl -v https://www.radiofrance.fr/personnes/michel-foucault
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/__init__.py", line 4, in <module>
    from .lazy_extractors import *
ModuleNotFoundError: No module named 'youtube_dl.extractor.lazy_extractors'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/youtube-dl", line 11, in <module>
    load_entry_point('youtube-dl==2021.12.17', 'console_scripts', 'youtube-dl')()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 490, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2854, in load_entry_point
    return ep.load()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2445, in load
    return self.resolve()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2451, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/lib/python3/dist-packages/youtube_dl/__init__.py", line 42, in <module>
    from .extractor import gen_extractors, list_extractors
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/__init__.py", line 9, in <module>
    from .extractors import *
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/extractors.py", line 70, in <module>
    from .arte import (
ImportError: cannot import name 'ArteTVCategoryIE' from 'youtube_dl.extractor.arte' (/usr/lib/python3/dist-packages/youtube_dl/extractor/arte.py)
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 153, in apport_excepthook
    with os.fdopen(os.open(pr_filename,
FileNotFoundError: [Errno 2] No such file or directory: '/var/crash/_usr_bin_youtube-dl.1000.crash'

Original exception was:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/__init__.py", line 4, in <module>
    from .lazy_extractors import *
ModuleNotFoundError: No module named 'youtube_dl.extractor.lazy_extractors'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/youtube-dl", line 11, in <module>
    load_entry_point('youtube-dl==2021.12.17', 'console_scripts', 'youtube-dl')()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 490, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2854, in load_entry_point
    return ep.load()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2445, in load
    return self.resolve()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2451, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/lib/python3/dist-packages/youtube_dl/__init__.py", line 42, in <module>
    from .extractor import gen_extractors, list_extractors
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/__init__.py", line 9, in <module>
    from .extractors import *
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/extractors.py", line 70, in <module>
    from .arte import (
ImportError: cannot import name 'ArteTVCategoryIE' from 'youtube_dl.extractor.arte' (/usr/lib/python3/dist-packages/youtube_dl/extractor/arte.py)

I searched for 'lazy_extractors' in the repository to see if I was missing a file lazy_extractors.py, for example. In vain.

In the result of the query: https://github.com/ytdl-org/youtube-dl/search?q=lazy_extractors, I saw the file youtube-dl/youtube_dl/extractor/__init__.py has these lines:

try:
    from .lazy_extractors import *
    from .lazy_extractors import _ALL_CLASSES
    _LAZY_LOADER = True
except ImportError:
    _LAZY_LOADER = False
    from .extractors import *

My local copy of youtube-dl/youtube_dl/extractor/__init__.py has these lines too.

What am I missing?

Any help would be greatly appreciated.

dirkf commented 1 year ago

After replacing the files, or now, run youtube-dl --version as admin. This regenerates the compiled (.pyc) files, which in your Python version are stored in a __pycache__ directory in each source directory.

What happened for you is that Python saw the extractor/__pycache__/extractors.cpython-38.pyc file. This old compiled file overrode extractor/extractors.py. Python would recompile the latter since it's newer, but because of permissions the new compiled file can't be written when yt-dl is run with a normal user account.

Looking at the code you quoted, your yt-dl was built with the option to create extractor/lazy_extractors.py disabled, so the except ImportError: suite was followed. The old compiled file references the ArteTVCategoryIE extractor that no longer exists in the PR branch, causing a second ImportError for the missing extractor.

The misleading backtrace would be avoided by s/t like this

_LAZY_LOADER = False
try:
    from .lazy_extractors import *
    from .lazy_extractors import _ALL_CLASSES
    _LAZY_LOADER = True
except ImportError:
    pass
if not _LAZY_LOADER:
    from .extractors import *
kdf8jsa1 commented 1 year ago

Thank you.

I ran $ sudo youtube-dl --version and got this: 

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/__init__.py", line 4, in <module>
    from .lazy_extractors import *
ModuleNotFoundError: No module named 'youtube_dl.extractor.lazy_extractors'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/youtube-dl", line 11, in <module>
    load_entry_point('youtube-dl==2021.12.17', 'console_scripts', 'youtube-dl')()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 490, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2854, in load_entry_point
    return ep.load()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2445, in load
    return self.resolve()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2451, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/lib/python3/dist-packages/youtube_dl/__init__.py", line 42, in <module>
    from .extractor import gen_extractors, list_extractors
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/__init__.py", line 9, in <module>
    from .extractors import *
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/extractors.py", line 70, in <module>
    from .arte import (
ImportError: cannot import name 'ArteTVCategoryIE' from 'youtube_dl.extractor.arte' (/usr/lib/python3/dist-packages/youtube_dl/extractor/arte.py)
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 153, in apport_excepthook
    with os.fdopen(os.open(pr_filename,
FileNotFoundError: [Errno 2] No such file or directory: '/var/crash/_usr_bin_youtube-dl.0.crash'

Original exception was:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/__init__.py", line 4, in <module>
    from .lazy_extractors import *
ModuleNotFoundError: No module named 'youtube_dl.extractor.lazy_extractors'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/youtube-dl", line 11, in <module>
    load_entry_point('youtube-dl==2021.12.17', 'console_scripts', 'youtube-dl')()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 490, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2854, in load_entry_point
    return ep.load()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2445, in load
    return self.resolve()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2451, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/lib/python3/dist-packages/youtube_dl/__init__.py", line 42, in <module>
    from .extractor import gen_extractors, list_extractors
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/__init__.py", line 9, in <module>
    from .extractors import *
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/extractors.py", line 70, in <module>
    from .arte import (
ImportError: cannot import name 'ArteTVCategoryIE' from 'youtube_dl.extractor.arte' (/usr/lib/python3/dist-packages/youtube_dl/extractor/arte.py)

I replaced line 3 to 11 of youtube-dl/youtube_dl/extractor/__init__.py with: 

_LAZY_LOADER = False
try:
    from .lazy_extractors import *
    from .lazy_extractors import _ALL_CLASSES
    _LAZY_LOADER = True
except ImportError:
    pass
if not _LAZY_LOADER:
    from .extractors import *

Ran $ sudo youtube-dl --version again and got this: 

Traceback (most recent call last):
  File "/usr/bin/youtube-dl", line 11, in <module>
    load_entry_point('youtube-dl==2021.12.17', 'console_scripts', 'youtube-dl')()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 490, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2854, in load_entry_point
    return ep.load()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2445, in load
    return self.resolve()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2451, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/lib/python3/dist-packages/youtube_dl/__init__.py", line 42, in <module>
    from .extractor import gen_extractors, list_extractors
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/__init__.py", line 11, in <module>
    from .extractors import *
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/extractors.py", line 70, in <module>
    from .arte import (
ImportError: cannot import name 'ArteTVCategoryIE' from 'youtube_dl.extractor.arte' (/usr/lib/python3/dist-packages/youtube_dl/extractor/arte.py)
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 153, in apport_excepthook
    with os.fdopen(os.open(pr_filename,
FileNotFoundError: [Errno 2] No such file or directory: '/var/crash/_usr_bin_youtube-dl.0.crash'

Original exception was:
Traceback (most recent call last):
  File "/usr/bin/youtube-dl", line 11, in <module>
    load_entry_point('youtube-dl==2021.12.17', 'console_scripts', 'youtube-dl')()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 490, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2854, in load_entry_point
    return ep.load()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2445, in load
    return self.resolve()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2451, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/lib/python3/dist-packages/youtube_dl/__init__.py", line 42, in <module>
    from .extractor import gen_extractors, list_extractors
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/__init__.py", line 11, in <module>
    from .extractors import *
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/extractors.py", line 70, in <module>
    from .arte import (
ImportError: cannot import name 'ArteTVCategoryIE' from 'youtube_dl.extractor.arte' (/usr/lib/python3/dist-packages/youtube_dl/extractor/arte.py)

I can see the "lazy_extractors" error is gone.

I searched Issues for:

Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 153, in apport_excepthook
    with os.fdopen(os.open(pr_filename,
FileNotFoundError: [Errno 2] No such file or directory: '/var/crash/_usr_bin_youtube-dl.0.crash'

and found #30416 but it seems unrelated.

How can I fix this error?

dirkf commented 1 year ago

Apparently we didn't hit it hard enough.

Please delete the contents of the __pycache__ directories under /usr/lib/python3/dist-packages/youtube_dl/; then re-run yt-dl as admin.

In the end (once it's working), revert __init__.py to avoid any possible future confusion.

kdf8jsa1 commented 1 year ago

Thank you again for the follow up. Still the same error i.e. Error in sys.excepthook unfortunately.

I deleted all files in the __pycache__ directory:

user@computer:/usr/lib/python3/dist-packages/youtube_dl/__pycache__$ sudo rm -i *
rm: remove regular file 'aes.cpython-38.pyc'? y
rm: remove regular file 'compat.cpython-38.pyc'? y
rm: remove regular file '__init__.cpython-38.pyc'? y
rm: remove regular file 'jsinterp.cpython-38.pyc'? y
rm: remove regular file 'options.cpython-38.pyc'? y
rm: remove regular file 'socks.cpython-38.pyc'? y
rm: remove regular file 'utils.cpython-38.pyc'? y
rm: remove regular file 'version.cpython-38.pyc'? y
user@computer:/usr/lib/python3/dist-packages/youtube_dl/__pycache__$ ls
user@computer:/usr/lib/python3/dist-packages/youtube_dl/__pycache__$ ll
total 8
drwxr-xr-x 2 root root 4096 Jan 20 21:24 ./
drwxr-xr-x 6 root root 4096 Sep 13 20:19 ../

Then ran yt-dl as admin:

$ sudo youtube-dl --version
Traceback (most recent call last):
  File "/usr/bin/youtube-dl", line 11, in <module>
    load_entry_point('youtube-dl==2021.12.17', 'console_scripts', 'youtube-dl')()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 490, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2854, in load_entry_point
    return ep.load()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2445, in load
    return self.resolve()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2451, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/lib/python3/dist-packages/youtube_dl/__init__.py", line 42, in <module>
    from .extractor import gen_extractors, list_extractors
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/__init__.py", line 11, in <module>
    from .extractors import *
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/extractors.py", line 70, in <module>
    from .arte import (
ImportError: cannot import name 'ArteTVCategoryIE' from 'youtube_dl.extractor.arte' (/usr/lib/python3/dist-packages/youtube_dl/extractor/arte.py)
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 153, in apport_excepthook
    with os.fdopen(os.open(pr_filename,
FileNotFoundError: [Errno 2] No such file or directory: '/var/crash/_usr_bin_youtube-dl.0.crash'

Original exception was:
Traceback (most recent call last):
  File "/usr/bin/youtube-dl", line 11, in <module>
    load_entry_point('youtube-dl==2021.12.17', 'console_scripts', 'youtube-dl')()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 490, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2854, in load_entry_point
    return ep.load()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2445, in load
    return self.resolve()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2451, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/lib/python3/dist-packages/youtube_dl/__init__.py", line 42, in <module>
    from .extractor import gen_extractors, list_extractors
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/__init__.py", line 11, in <module>
    from .extractors import *
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/extractors.py", line 70, in <module>
    from .arte import (
ImportError: cannot import name 'ArteTVCategoryIE' from 'youtube_dl.extractor.arte' (/usr/lib/python3/dist-packages/youtube_dl/extractor/arte.py)

(which repopulated the __pycache__ directory with all the files I had deleted. I reiterated the above. In vain.)

Sorry. What else would you suggest I try? Would sharing the content of other file(s) help you?

dirkf commented 1 year ago

Critically, do the same with /usr/lib/python3/dist-packages/youtube_dl/extractor/__pycache__: there are cache directories in all the other source directories but the error is from the extractor directory.

kdf8jsa1 commented 1 year ago

I still get an error message (see at the bottom).

I actually started from scratch. sudo apt-get remove youtube-dl and then installed yt-dl again and did the following:

Replaced the files radiofrance.py, extractors.py with files from PR and removed the files franceinter.py franceculture.py as in PR.

Updated /usr/lib/python3/dist-packages/youtube_dl/extractor/__init__.py with:

_LAZY_LOADER = False
try:
    from .lazy_extractors import *
    from .lazy_extractors import _ALL_CLASSES
    _LAZY_LOADER = True
except ImportError:
    pass
if not _LAZY_LOADER:
    from .extractors import *

as suggested in https://github.com/ytdl-org/youtube-dl/issues/31464#issuecomment-1398591332.

Listed all .pyc files:

/usr/lib/python3/dist-packages/youtube_dl$ find . -name '*.pyc' -type f -print 
./__pycache__/__init__.cpython-38.pyc
./__pycache__/version.cpython-38.pyc
./__pycache__/compat.cpython-38.pyc
./__pycache__/options.cpython-38.pyc
./__pycache__/aes.cpython-38.pyc
./__pycache__/utils.cpython-38.pyc
./__pycache__/socks.cpython-38.pyc
./__pycache__/jsinterp.cpython-38.pyc
./extractor/__pycache__/anvato.cpython-38.pyc
./extractor/__pycache__/__init__.cpython-38.pyc
./extractor/__pycache__/once.cpython-38.pyc
./extractor/__pycache__/adn.cpython-38.pyc
./extractor/__pycache__/amara.cpython-38.pyc
./extractor/__pycache__/extractors.cpython-38.pyc
./extractor/__pycache__/americastestkitchen.cpython-38.pyc
./extractor/__pycache__/yahoo.cpython-38.pyc
./extractor/__pycache__/aenetworks.cpython-38.pyc
./extractor/__pycache__/amp.cpython-38.pyc
./extractor/__pycache__/turner.cpython-38.pyc
./extractor/__pycache__/animeondemand.cpython-38.pyc
./extractor/__pycache__/aliexpress.cpython-38.pyc
./extractor/__pycache__/aol.cpython-38.pyc
./extractor/__pycache__/adobepass.cpython-38.pyc
./extractor/__pycache__/airmozilla.cpython-38.pyc
./extractor/__pycache__/afreecatv.cpython-38.pyc
./extractor/__pycache__/abcotvs.cpython-38.pyc
./extractor/__pycache__/amcnetworks.cpython-38.pyc
./extractor/__pycache__/abcnews.cpython-38.pyc
./extractor/__pycache__/adultswim.cpython-38.pyc
./extractor/__pycache__/brightcove.cpython-38.pyc
./extractor/__pycache__/academicearth.cpython-38.pyc
./extractor/__pycache__/adobeconnect.cpython-38.pyc
./extractor/__pycache__/aljazeera.cpython-38.pyc
./extractor/__pycache__/adobetv.cpython-38.pyc
./extractor/__pycache__/allocine.cpython-38.pyc
./extractor/__pycache__/common.cpython-38.pyc
./extractor/__pycache__/alphaporno.cpython-38.pyc
./extractor/__pycache__/vimeo.cpython-38.pyc
./extractor/__pycache__/youtube.cpython-38.pyc
./extractor/__pycache__/acast.cpython-38.pyc
./extractor/__pycache__/abc.cpython-38.pyc
./extractor/__pycache__/theplatform.cpython-38.pyc
./downloader/__pycache__/http.cpython-38.pyc
./downloader/__pycache__/__init__.cpython-38.pyc
./downloader/__pycache__/external.cpython-38.pyc
./downloader/__pycache__/dash.cpython-38.pyc
./downloader/__pycache__/rtsp.cpython-38.pyc
./downloader/__pycache__/common.cpython-38.pyc
./downloader/__pycache__/rtmp.cpython-38.pyc
./downloader/__pycache__/hls.cpython-38.pyc
./downloader/__pycache__/ism.cpython-38.pyc
./downloader/__pycache__/f4m.cpython-38.pyc
./downloader/__pycache__/fragment.cpython-38.pyc
./postprocessor/__pycache__/__init__.cpython-38.pyc
./postprocessor/__pycache__/execafterdownload.cpython-38.pyc
./postprocessor/__pycache__/xattrpp.cpython-38.pyc
./postprocessor/__pycache__/ffmpeg.cpython-38.pyc
./postprocessor/__pycache__/embedthumbnail.cpython-38.pyc
./postprocessor/__pycache__/metadatafromtitle.cpython-38.pyc
./postprocessor/__pycache__/common.cpython-38.pyc

Deleted all .pyc files:

$ find . -name '*.pyc' -type f -print | xargs sudo rm

Ran: 

$ sudo youtube-dl --version
Traceback (most recent call last):
  File "/usr/bin/youtube-dl", line 11, in <module>
    load_entry_point('youtube-dl==2021.6.6', 'console_scripts', 'youtube-dl')()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 490, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2854, in load_entry_point
    return ep.load()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2445, in load
    return self.resolve()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2451, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/lib/python3/dist-packages/youtube_dl/__init__.py", line 42, in <module>
    from .extractor import gen_extractors, list_extractors
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/__init__.py", line 11, in <module>
    from .extractors import *
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/extractors.py", line 54, in <module>
    from .alsace20tv import (
ModuleNotFoundError: No module named 'youtube_dl.extractor.alsace20tv'
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 153, in apport_excepthook
    with os.fdopen(os.open(pr_filename,
FileNotFoundError: [Errno 2] No such file or directory: '/var/crash/_usr_bin_youtube-dl.0.crash'

Original exception was:
Traceback (most recent call last):
  File "/usr/bin/youtube-dl", line 11, in <module>
    load_entry_point('youtube-dl==2021.6.6', 'console_scripts', 'youtube-dl')()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 490, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2854, in load_entry_point
    return ep.load()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2445, in load
    return self.resolve()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2451, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/lib/python3/dist-packages/youtube_dl/__init__.py", line 42, in <module>
    from .extractor import gen_extractors, list_extractors
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/__init__.py", line 11, in <module>
    from .extractors import *
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/extractors.py", line 54, in <module>
    from .alsace20tv import (
ModuleNotFoundError: No module named 'youtube_dl.extractor.alsace20tv'

Any idea?

dirkf commented 1 year ago

Sorry, of course other changes since the release have affected the version of extractors/extractors.py on which the PR is based and it expects some modules that aren't in the release.

Try this version that I've re-based against the release with only the RF changes (remove the .txt when saving): extractors.py

kdf8jsa1 commented 1 year ago

Thank you. There is some improvement.

I downloaded extractors.py and replaced my local version of the file, then ran youtube-dl as admin which produced no error this time:

$ sudo mv extractors.py /usr/lib/python3/dist-packages/youtube_dl/extractor/extractors.py
$ sudo youtube-dl --version
2021.06.06

Then I tried to download the radio shows/programmes from Radio France's website but got an error:

$ youtube-dl -v https://www.radiofrance.fr/personnes/michel-foucault
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://www.radiofrance.fr/personnes/michel-foucault']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.06.06
[debug] Python version 3.8.10 (CPython) - Linux-5.4.0-131-generic-x86_64-with-glibc2.29
[debug] exe versions: ffmpeg 4.2.7, ffprobe 4.2.7, rtmpdump 2.4
[debug] Proxy map: {}
[generic] michel-foucault: Requesting header
WARNING: Falling back on generic information extractor.
[generic] michel-foucault: Downloading webpage
[generic] michel-foucault: Extracting information
ERROR: Unsupported URL: https://www.radiofrance.fr/personnes/michel-foucault
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/youtube_dl/YoutubeDL.py", line 815, in wrapper
    return func(self, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/youtube_dl/YoutubeDL.py", line 836, in __extract_info
    ie_result = ie.extract(url)
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/common.py", line 534, in extract
    ie_result = self._real_extract(url)
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/generic.py", line 3520, in _real_extract
    raise UnsupportedError(url)
youtube_dl.utils.UnsupportedError: Unsupported URL: https://www.radiofrance.fr/personnes/michel-foucault

I searched for the lines after Traceback in Issues but nothing came up.

What can I try?

Thanks again for your help.

dirkf commented 1 year ago

Try getting a single programme from that page, eg https://www.radiofrance.fr/franceculture/podcasts/le-pourquoi-du-comment-economie-et-social/le-neo-liberalisme-et-la-fin-du-plein-emploi-6657160. The PR doesn't yet handle playlists.

kdf8jsa1 commented 1 year ago

I tried but got an error message which I found in https://github.com/ytdl-org/youtube-dl/issues/30251#issuecomment-974714112 where you suggest to run the command with --write-page; so I did (if it helps):

$ youtube-dl -v https://www.radiofrance.fr/franceculture/podcasts/le-pourquoi-du-comment-economie-et-social/le-neo-liberalisme-et-la-fin-du-plein-emploi-6657160 --write-page
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://www.radiofrance.fr/franceculture/podcasts/le-pourquoi-du-comment-economie-et-social/le-neo-liberalisme-et-la-fin-du-plein-emploi-6657160', '--write-page']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.06.06
[debug] Python version 3.8.10 (CPython) - Linux-5.4.0-131-generic-x86_64-with-glibc2.29
[debug] exe versions: ffmpeg 4.2.7, ffprobe 4.2.7, rtmpdump 2.4
[debug] Proxy map: {}
[RadioFrancePodcastEpisode] 6657160: Downloading webpage
[RadioFrancePodcastEpisode] Saving request to 6657160_https_-_www.radiofrance.fr_franceculture_podcasts_le-pourquoi-du-comment-economie-et-social_le-neo-liberalisme-et-la-fin-du-plein-emploi-6657160.dump
ERROR: Unable to extract API data; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/youtube_dl/YoutubeDL.py", line 815, in wrapper
    return func(self, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/youtube_dl/YoutubeDL.py", line 836, in __extract_info
    ie_result = ie.extract(url)
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/common.py", line 534, in extract
    ie_result = self._real_extract(url)
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/radiofrance.py", line 174, in _real_extract
    webpage, api_data = self.get_data(url, 'path', id)
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/radiofrance.py", line 129, in get_data
    api_data = self.extract_api_data(api_path, id, webpage)
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/radiofrance.py", line 22, in extract_api_data
    json = self._search_regex(pattern, html, 'API data', flags=re.DOTALL, group='json')
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/common.py", line 1012, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
youtube_dl.utils.RegexNotFoundError: Unable to extract API data; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

6657160https-_www.radiofrance.fr_franceculture_podcasts_le-pourquoi-du-comment-economie-et-social_le-neo-liberalisme-et-la-fin-du-plein-emploi-6657160.dump.gz

(Re playlist: https://github.com/ytdl-org/youtube-dl/pull/31435#issuecomment-1399668067. Thanks. Yes, that would great if yt-dl manages to download playlist on www.radiofrance.com)

dirkf commented 1 year ago

I'll let the author debug the PR!

kdf8jsa1 commented 1 year ago

Noted. I will keep an eye on the PR. Thank you for posting https://github.com/ytdl-org/youtube-dl/pull/31435#issuecomment-1400967079https://github.com/ytdl-org/youtube-dl/pull/31435#issuecomment-1400967079. Looking forward.

Thank you for your help again.