Error when running in a google cloud function with Python 3.7

x955199 commented 4 years ago

Checklist

[x] I'm reporting a broken site support issue
[x] I've verified that I'm running youtube-dl version 2020.03.24
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar bug reports including closed ones
[x] I've read bugs section in FAQ

Verbose log

"Traceback (most recent call last):
  File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 346, in run_http_function
    result = _function_handler.invoke_user_function(flask.request)
  File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 217, in invoke_user_function
    return call_user_function(request_or_event)
  File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 210, in call_user_function
    return self._user_function(request_or_event)
  File "/user_code/main.py", line 68, in main
    videoId = videoToAudio(request_json['url'])
  File "/user_code/main.py", line 54, in videoToAudio
    r = ydl.extract_info(url, download=False)
  File "/env/local/lib/python3.7/site-packages/youtube_dl/YoutubeDL.py", line 797, in extract_info
    ie_result = ie.extract(url)
  File "/env/local/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 530, in extract
    ie_result = self._real_extract(url)
  File "/env/local/lib/python3.7/site-packages/youtube_dl/extractor/youtube.py", line 1681, in _real_extract
    video_webpage = self._download_webpage(url, video_id)
  File "/env/local/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 794, in _download_webpage
    expected_status=expected_status)
  File "/env/local/lib/python3.7/site-packages/youtube_dl/extractor/youtube.py", line 272, in _download_webpage_handle
    *args, **compat_kwargs(kwargs))
  File "/env/local/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 660, in _download_webpage_handle
    urlh = self._request_webpage(url_or_request, video_id, note, errnote, fatal, data=data, headers=headers, query=query, expected_status=expected_status)
  File "/env/local/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 602, in _request_webpage
    self.report_download_webpage(video_id)
  File "/env/local/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 922, in report_download_webpage
    self.to_screen('%s: Downloading webpage' % video_id)
  File "/env/local/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 914, in to_screen
    self._downloader.to_screen('[%s] %s' % (self.IE_NAME, msg))
  File "/env/local/lib/python3.7/site-packages/youtube_dl/YoutubeDL.py", line 503, in to_screen
    return self.to_stdout(message, skip_eol, check_quiet=True)
  File "/env/local/lib/python3.7/site-packages/youtube_dl/YoutubeDL.py", line 517, in to_stdout
    self._write_string(output, self._screen_file)
  File "/env/local/lib/python3.7/site-packages/youtube_dl/YoutubeDL.py", line 506, in _write_string
    write_string(s, out=out, encoding=self.params.get('encoding'))
  File "/env/local/lib/python3.7/site-packages/youtube_dl/utils.py", line 3102, in write_string
    out.buffer.write(byt)
TypeError: string argument expected, got 'bytes'

Description

Getting this error when running in a google cloud function with Python 3.7, the code works when I run it on my local machine with Python 2.X. My Code:


   options = {
    'format': 'bestaudio/best',
    'outtmpl': '/tmp/%(id)s.%(ext)s',
    postprocessors': [{
     'key': 'FFmpegExtractAudio',
     'preferredcodec': 'wav'
    }]
   }

   with youtube_dl.YoutubeDL(options) as ydl:
      r = ydl.extract_info(url, download=False)
      ydl.download([url])

This issue was reported before here https://github.com/ytdl-org/youtube-dl/issues/22549, but reporting again since I believe it is still happening. After looking into it for a little bit, it seems like for Python 2.x it works cause there is a special condition for it. Not sure what is the expected behavior for Python 3.7.

https://github.com/ytdl-org/youtube-dl/blob/75294a5ed03f4443970478f3f4eac572239cec45/youtube_dl/utils.py#L3096

dstftw commented 4 years ago

https://github.com/ytdl-org/youtube-dl/issues/22549#issuecomment-536338664

x955199 commented 4 years ago

@dstftw I saw your comment, but not sure if there is a solution for it?

UPDATE: works on my machine with Python 3.7, maybe an issue within google cloud function?

steveobd commented 4 years ago

this is fairly easy to replicate. why is it closed?

ytdl-org / youtube-dl