Closed chiragkanzariya-simformsolutions closed 3 years ago
Hi; Thanks for opening an issue.
Seems to be working fine on my end for a couple Youtube URLs I'm testing for.
Could you give me a URL this fails for?
The youtube request is using the builtin urllib
library instead of requests
, I have a feeling that may be causing the error here.
I pushed an update to the submodule, could you try testing this by doing:
pip uninstall -y url_metadata
git clone 'https://github.com/seanbreckenridge/url_metadata'
cd ./url_metadata
git submodule update --init
pip install --user .
python3 -m url_metadata get "some youtube url"
@seanbreckenridge This is a youtube URL I have tried: https://www.youtube.com/watch?v=6G0mablNVXY
But yeah I think I haven't checked with urllib
so let me try with that.
Thank you for your quick response.
That URL works fine me (though I don't see any subtitles on that video, not sure if this is a region issue), let me know if the above works.
I have attached my error here:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/urllib/request.py", line 1350, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/http/client.py", line 1255, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/http/client.py", line 1301, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/http/client.py", line 1250, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/http/client.py", line 1010, in _send_output
self.send(msg)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/http/client.py", line 950, in send
self.connect()
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/http/client.py", line 1424, in connect
self.sock = self._context.wrap_socket(self.sock,
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/ssl.py", line 500, in wrap_socket
return self.sslsocket_class._create(
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/ssl.py", line 1040, in _create
self.do_handshake()
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/ssl.py", line 1309, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1123)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "get_metadata.py", line 8, in <module>
m = metadata("https://www.youtube.com/watch?v=6G0mablNVXY")
File "/Users/chirag.kanzariya/Desktop/360View/test_env/lib/python3.8/site-packages/url_metadata/__init__.py", line 13, in metadata
return default_cache.get(url)
File "/Users/chirag.kanzariya/Desktop/360View/test_env/lib/python3.8/site-packages/url_metadata/core.py", line 120, in get
data: Metadata = self.request_data(uurl)
File "/Users/chirag.kanzariya/Desktop/360View/test_env/lib/python3.8/site-packages/url_metadata/core.py", line 161, in request_data
metadata.subtitles = download_subtitles(
File "/Users/chirag.kanzariya/Desktop/360View/test_env/lib/python3.8/site-packages/url_metadata/youtube.py", line 23, in download_subtitles
return download_subs(youtube_id, lang)
File "/Users/chirag.kanzariya/Desktop/360View/test_env/lib/python3.8/site-packages/url_metadata/yt_subs/subtitles_downloader.py", line 22, in download_subs
video_info: Dict[str, Any] = get_video_info(video_identifier)
File "/Users/chirag.kanzariya/Desktop/360View/test_env/lib/python3.8/site-packages/url_metadata/yt_subs/subtitles_downloader.py", line 38, in get_video_info
resp: HTTPResponse = urllib.request.urlopen(
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/urllib/request.py", line 525, in open
response = self._open(req, data)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/urllib/request.py", line 542, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/urllib/request.py", line 1393, in https_open
return self.do_open(http.client.HTTPSConnection, req,
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/urllib/request.py", line 1353, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1123)>
Should get away from using urllib
in general, so I pushed a new version (0.1.4) to pypi. Can you try:
pip uninstall -y url_metadata
pip install -U url_metadata
and try running it with the same URL?
This the code that I am trying with your library:
from url_metadata import metadata
m = metadata("https://www.youtube.com/watch?v=6G0mablNVXY")
print(m.info)
print(m.info["images"][0]["src"])
print('--------')
len(m.info["images"])
print(m.info["title"])
print('--------')
m.info["title"]
print(m.info["description"])
And BTW above steps also giving me the same error.
Hmm, alright.
I'll have to look further into the SSL error.
Its quite late for me so I will probably take a look at this further tomorrow
Just to confirm, does this video have subtitles for you or not? I don't see any subtitles on it.
Okay no worry, Thanks for the update.
I just want a title, description, and image of the URL. I am not interested in the subtitle for now.
Right, but the error is coming from trying to download the subtitles.
If I'm not able to figure out the SSL error (honestly have no clue, but it seems it may be an error with the python installation process? see here for lots of similar errors/a possible solution), I may resolve this by adding a flag to skip downloading subtitles, or just reporting/warning if theres a requests error, instead of crashing.
Pushed some changes to master. Before I push a version, could I get you to run:
pip uninstall -y url_metadata
git clone 'https://github.com/seanbreckenridge/url_metadata'
cd ./url_metadata
git submodule update --init
pip install --user .
python3 -m url_metadata get "some youtube url"
again and see if it ignores the error now?
Else you can do:
import logging
from url_metadata import URLMetadataCache
# pass the skip_subtitles flag so it doesnt download those
cache = URLMetadataCache(loglevel=logging.DEBUG, skip_subtitles=True)
c = cache.get("some youtube url")
or with the flag:
python3 -m url_metadata --skip-subtitles get "some youtube url"
I have some work in my bucket, I will check after 2 hours and let you know the details
have pushed a new version (v0.1.5) to pypi with a flag/kwarg to ignore subtitles, feel free to re-open this if the flag doesn't work for you
It maybe looks like the library is not properly configured with SSL. Here I am getting the below error :
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1123)>
I am getting above both errors when I am parsing any YouTube URL.