[GoogleDrive:Folder] Extractor returning 400 Bad Request. Error: Expected string or bytes-like object

[justshrey]

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

• [X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

• [X] I'm reporting that yt-dlp is broken on a supported site
• [X] I've verified that I have updated yt-dlp to nightly or master (update instructions)
• [X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
• [X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
• [X] I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
• [X] I've read the guidelines for opening an issue
• [X] I've read about sharing account credentials and I'm willing to share it if required

Region

India

Provide a description that is worded well enough to be understood

This issue occurs even with the test url ( https://drive.google.com/drive/folders/1dQ4sx0-__Nvg65rxTSgQrl7VyW_FZ9QI ) present in the GoogleDrive.py

Have confirmed with the latest nightly version

Provide verbose output that clearly demonstrates the problem

• [X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
• [ ] If using API, add 'verbose': True to YoutubeDL params instead
• [X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

[debug] Command-line config: ['-vU', 'https://drive.google.com/drive/folders/1dQ4sx0-__Nvg65rxTSgQrl7VyW_FZ9QI']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version nightly@2024.07.15.232803 from yt-dlp/yt-dlp-nightly-builds [39e6c4cb4] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19045-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg 7.0-full_build-www.gyan.dev (setts), ffprobe 7.0-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.20.0, brotli-1.1.0, certifi-2024.07.04, curl_cffi-0.5.10, mutagen-1.47.0, requests-2.32.3, sqlite3-3.35.5, urllib3-2.2.2, websockets-12.0
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets, curl_cffi
[debug] Loaded 1829 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp-nightly-builds/releases/latest
Latest version: nightly@2024.07.15.232803 from yt-dlp/yt-dlp-nightly-builds
yt-dlp is up to date (nightly@2024.07.15.232803 from yt-dlp/yt-dlp-nightly-builds)
[GoogleDrive:Folder] Extracting URL: https://drive.google.com/drive/folders/1dQ4sx0-__Nvg65rxTSgQrl7VyW_FZ9QI
[GoogleDrive:Folder] 1dQ4sx0-__Nvg65rxTSgQrl7VyW_FZ9QI: Downloading webpage
[GoogleDrive:Folder] 1dQ4sx0-__Nvg65rxTSgQrl7VyW_FZ9QI: Downloading webpage
WARNING: [GoogleDrive:Folder] Unable to download webpage: HTTP Error 400: Bad Request
ERROR: expected string or bytes-like object
Traceback (most recent call last):
  File "yt_dlp\YoutubeDL.py", line 1622, in wrapper
  File "yt_dlp\YoutubeDL.py", line 1757, in __extract_info
  File "yt_dlp\extractor\common.py", line 740, in extract
  File "yt_dlp\extractor\googledrive.py", line 339, in _real_extract
  File "yt_dlp\extractor\googledrive.py", line 323, in _call_api
  File "yt_dlp\extractor\common.py", line 1347, in _search_json
  File "yt_dlp\extractor\common.py", line 1313, in _search_regex
  File "re.py", line 201, in search
TypeError: expected string or bytes-like object

[milosnikic]

@justshrey This seems like you are providing None in regex search function.

[seproDev]

They are providing nothing. This is a bug in the extractor code. It will need to be adjusted for the new API endpoint.

[grqz]

They are providing nothing. This is a bug in the extractor code. It will need to be adjusted for the new API endpoint.

Seems like the video ids are embedded in the page source. (extracted from view-source:https://drive.google.com/drive/folders/1dQ4sx0-__Nvg65rxTSgQrl7VyW_FZ9QI, 1eE-5jm2G8N5FL4HHeexrL0-Lg15bc0-D is the video id)

The site worked fine when I blocked clients6.google.com, play.google.com and googleapis.com(all XHR GET requests are blocked) with firefox.

I don't know if there's an unused API endpoint

[grqz]

Something I found in the page source:

Details(very long!) ```jsonc [ "driveweb;XVZmSc;HMxWxf[[\"1dQ4sx0-__Nvg65rxTSgQrl7VyW_FZ9QI\"],[4,1,1],1]", [ [ "driveweb;XVZmSc;HMxWxf[[\"1dQ4sx0-__Nvg65rxTSgQrl7VyW_FZ9QI\"],[4,1,1],1]", [ [ "1eE-5jm2G8N5FL4HHeexrL0-Lg15bc0-D", //... ], [ "1v2hFMjnxl8WSjiYIThq0rPscoLb8zwYy", //... ], [ "1Jp0I0tS-qMxtXNehGQW5_hWhwgC0FeeB", //... ] ], false, null, "Files" ] ], false, null, [ [ "1eE-5jm2G8N5FL4HHeexrL0-Lg15bc0-D", //... ], [ "1v2hFMjnxl8WSjiYIThq0rPscoLb8zwYy", //... ], [ "1Jp0I0tS-qMxtXNehGQW5_hWhwgC0FeeB", //... ] ] ] ```

```json [ "driveweb;XVZmSc;HMxWxf[[\"1dQ4sx0-__Nvg65rxTSgQrl7VyW_FZ9QI\"],[4,1,1],1]", [ [ "driveweb;XVZmSc;HMxWxf[[\"1dQ4sx0-__Nvg65rxTSgQrl7VyW_FZ9QI\"],[4,1,1],1]", [ [ "1eE-5jm2G8N5FL4HHeexrL0-Lg15bc0-D", [ "1dQ4sx0-__Nvg65rxTSgQrl7VyW_FZ9QI" ], "A Walk through our forest. [kQot7nZLeJo].webm", "video/webm", false, null, false, false, false, 1655151526612, 1571773704000, null, null, 338040226, [ [ 1, "14714185515644744399", null, null, null, null, false ] ], null, [ 1, "14714185515644744399", null, null, null, null, false ], null, null, null, "https://drive-thirdparty.googleusercontent.com/32/type/video/webm", true, null, null, 2, false, null, 338040226, null, false, false, null, null, null, null, false, null, null, null, null, null, null, null, true, "webm", null, null, null, null, null, null, null, null, null, null, null, null, null, null, [ [ 1 ] ], "691301496089", null, null, null, null, null, null, null, null, 1571773704000, 3, null, null, null, 14, null, null, null, null, null, null, null, null, [ false, null, false, false, true, false, false, false, false, null, null, false, null, null, false, null, null, null, null, false, false, null, null, null, null, null, false, null, null, null, null, null, null, null, null, false, false, false, null, false, true, null, null, true, false, null, false, null, null, false, null, false, false, false, false, null, null, null, true, null, null, false, null, false, true, false ], null, null, null, null, null, 1, null, null, null, false, null, null, null, null, null, null, null, null, null, null, null, null, null, false, null, null, null, null, null, null, "https://drive.google.com/file/d/1eE-5jm2G8N5FL4HHeexrL0-Lg15bc0-D/view?usp\u003ddrive_web", null, null, null, null, null, null, null, null, null, [ [ false ] ], null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, false, null, null, null, [ null, null, 0, false, false, false ], null, null, null, 0, null, null, null, null, null, null, null, false ], [ "1v2hFMjnxl8WSjiYIThq0rPscoLb8zwYy", [ "1dQ4sx0-__Nvg65rxTSgQrl7VyW_FZ9QI" ], "Bryce Canyon National Park [7uKucJejmTc].webm", "video/webm", false, null, false, false, false, 1655151690290, 1565879180000, null, null, 86579536, [ [ 1, "14714185515644744399", null, null, null, null, false ] ], null, [ 1, "14714185515644744399", null, null, null, null, false ], null, null, null, "https://drive-thirdparty.googleusercontent.com/32/type/video/webm", true, null, null, 2, false, null, 86579536, null, false, false, null, null, null, null, false, null, null, null, null, null, null, null, true, "webm", null, null, null, null, null, null, null, null, null, null, null, null, null, null, [ [ 1 ] ], "691301496089", null, null, null, null, null, null, null, null, 1565879180000, 3, null, null, null, 10, null, null, null, null, null, null, null, null, [ false, null, false, false, true, false, false, false, false, null, null, false, null, null, false, null, null, null, null, false, false, null, null, null, null, null, false, null, null, null, null, null, null, null, null, false, false, false, null, false, true, null, null, true, false, null, false, null, null, false, null, false, false, false, false, null, null, null, true, null, null, false, null, false, true, false ], null, null, null, null, null, 1, null, null, null, false, null, null, null, null, null, null, null, null, null, null, null, null, null, false, null, null, null, null, null, null, "https://drive.google.com/file/d/1v2hFMjnxl8WSjiYIThq0rPscoLb8zwYy/view?usp\u003ddrive_web", null, null, null, null, null, null, null, null, null, [ [ false ] ], null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, false, null, null, null, [ null, null, 0, false, false, false ], null, null, null, 0, null, null, null, null, null, null, null, false ], [ "1Jp0I0tS-qMxtXNehGQW5_hWhwgC0FeeB", [ "1dQ4sx0-__Nvg65rxTSgQrl7VyW_FZ9QI" ], "Forrest walk with Best Mind Refresh Music Mithran [tEvJKrE4cS0].webm", "video/webm", false, null, false, false, false, 1655151896280, 1634932741000, null, null, 98257300, [ [ 1, "14714185515644744399", null, null, null, null, false ] ], null, [ 1, "14714185515644744399", null, null, null, null, false ], null, null, null, "https://drive-thirdparty.googleusercontent.com/32/type/video/webm", true, null, null, 2, false, null, 98257300, null, false, false, null, null, null, null, false, null, null, null, null, null, null, null, true, "webm", null, null, null, null, null, null, null, null, null, null, null, null, null, null, [ [ 1 ] ], "691301496089", null, null, null, null, null, null, null, null, 1634932741000, 3, null, null, null, 11, null, null, null, null, null, null, null, null, [ false, null, false, false, true, false, false, false, false, null, null, false, null, null, false, null, null, null, null, false, false, null, null, null, null, null, false, null, null, null, null, null, null, null, null, false, false, false, null, false, true, null, null, true, false, null, false, null, null, false, null, false, false, false, false, null, null, null, true, null, null, false, null, false, true, false ], null, null, null, null, null, 1, null, null, null, false, null, null, null, null, null, null, null, null, null, null, null, null, null, false, null, null, null, null, null, null, "https://drive.google.com/file/d/1Jp0I0tS-qMxtXNehGQW5_hWhwgC0FeeB/view?usp\u003ddrive_web", null, null, null, null, null, null, null, null, null, [ [ false ] ], null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, false, null, null, null, [ null, null, 0, false, false, false ], null, null, null, 0, null, null, null, null, null, null, null, false ] ], false, null, "Files" ] ], false, null, [ [ "1eE-5jm2G8N5FL4HHeexrL0-Lg15bc0-D", [ "1dQ4sx0-__Nvg65rxTSgQrl7VyW_FZ9QI" ], "A Walk through our forest. [kQot7nZLeJo].webm", "video/webm", false, null, false, false, false, 1655151526612, 1571773704000, null, null, 338040226, [ [ 1, "14714185515644744399", null, null, null, null, false ] ], null, [ 1, "14714185515644744399", null, null, null, null, false ], null, null, null, "https://drive-thirdparty.googleusercontent.com/32/type/video/webm", true, null, null, 2, false, null, 338040226, null, false, false, null, null, null, null, false, null, null, null, null, null, null, null, true, "webm", null, null, null, null, null, null, null, null, null, null, null, null, null, null, [ [ 1 ] ], "691301496089", null, null, null, null, null, null, null, null, 1571773704000, 3, null, null, null, 14, null, null, null, null, null, null, null, null, [ false, null, false, false, true, false, false, false, false, null, null, false, null, null, false, null, null, null, null, false, false, null, null, null, null, null, false, null, null, null, null, null, null, null, null, false, false, false, null, false, true, null, null, true, false, null, false, null, null, false, null, false, false, false, false, null, null, null, true, null, null, false, null, false, true, false ], null, null, null, null, null, 1, null, null, null, false, null, null, null, null, null, null, null, null, null, null, null, null, null, false, null, null, null, null, null, null, "https://drive.google.com/file/d/1eE-5jm2G8N5FL4HHeexrL0-Lg15bc0-D/view?usp\u003ddrive_web", null, null, null, null, null, null, null, null, null, [ [ false ] ], null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, false, null, null, null, [ null, null, 0, false, false, false ], null, null, null, 0, null, null, null, null, null, null, null, false ], [ "1v2hFMjnxl8WSjiYIThq0rPscoLb8zwYy", [ "1dQ4sx0-__Nvg65rxTSgQrl7VyW_FZ9QI" ], "Bryce Canyon National Park [7uKucJejmTc].webm", "video/webm", false, null, false, false, false, 1655151690290, 1565879180000, null, null, 86579536, [ [ 1, "14714185515644744399", null, null, null, null, false ] ], null, [ 1, "14714185515644744399", null, null, null, null, false ], null, null, null, "https://drive-thirdparty.googleusercontent.com/32/type/video/webm", true, null, null, 2, false, null, 86579536, null, false, false, null, null, null, null, false, null, null, null, null, null, null, null, true, "webm", null, null, null, null, null, null, null, null, null, null, null, null, null, null, [ [ 1 ] ], "691301496089", null, null, null, null, null, null, null, null, 1565879180000, 3, null, null, null, 10, null, null, null, null, null, null, null, null, [ false, null, false, false, true, false, false, false, false, null, null, false, null, null, false, null, null, null, null, false, false, null, null, null, null, null, false, null, null, null, null, null, null, null, null, false, false, false, null, false, true, null, null, true, false, null, false, null, null, false, null, false, false, false, false, null, null, null, true, null, null, false, null, false, true, false ], null, null, null, null, null, 1, null, null, null, false, null, null, null, null, null, null, null, null, null, null, null, null, null, false, null, null, null, null, null, null, "https://drive.google.com/file/d/1v2hFMjnxl8WSjiYIThq0rPscoLb8zwYy/view?usp\u003ddrive_web", null, null, null, null, null, null, null, null, null, [ [ false ] ], null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, false, null, null, null, [ null, null, 0, false, false, false ], null, null, null, 0, null, null, null, null, null, null, null, false ], [ "1Jp0I0tS-qMxtXNehGQW5_hWhwgC0FeeB", [ "1dQ4sx0-__Nvg65rxTSgQrl7VyW_FZ9QI" ], "Forrest walk with Best Mind Refresh Music Mithran [tEvJKrE4cS0].webm", "video/webm", false, null, false, false, false, 1655151896280, 1634932741000, null, null, 98257300, [ [ 1, "14714185515644744399", null, null, null, null, false ] ], null, [ 1, "14714185515644744399", null, null, null, null, false ], null, null, null, "https://drive-thirdparty.googleusercontent.com/32/type/video/webm", true, null, null, 2, false, null, 98257300, null, false, false, null, null, null, null, false, null, null, null, null, null, null, null, true, "webm", null, null, null, null, null, null, null, null, null, null, null, null, null, null, [ [ 1 ] ], "691301496089", null, null, null, null, null, null, null, null, 1634932741000, 3, null, null, null, 11, null, null, null, null, null, null, null, null, [ false, null, false, false, true, false, false, false, false, null, null, false, null, null, false, null, null, null, null, false, false, null, null, null, null, null, false, null, null, null, null, null, null, null, null, false, false, false, null, false, true, null, null, true, false, null, false, null, null, false, null, false, false, false, false, null, null, null, true, null, null, false, null, false, true, false ], null, null, null, null, null, 1, null, null, null, false, null, null, null, null, null, null, null, null, null, null, null, null, null, false, null, null, null, null, null, null, "https://drive.google.com/file/d/1Jp0I0tS-qMxtXNehGQW5_hWhwgC0FeeB/view?usp\u003ddrive_web", null, null, null, null, null, null, null, null, null, [ [ false ] ], null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, false, null, null, null, [ null, null, 0, false, false, false ], null, null, null, 0, null, null, null, null, null, null, null, false ] ] ] ```

Maybe _search_json will work

[atom-smasher]

Any updates? I'm trying to download a video without sharing enabled, from a "folder" without sharing enabled. The only link I can find is for the folder, not the video.

WARNING: [GoogleDrive:Folder] Unable to download webpage: HTTP Error 400: Bad Request
ERROR: expected string or bytes-like object

Any workarounds for getting a link for the video?

yt-dlp: master@2024.11.15.225212

[atom-smasher]

[!CAUTION] This comment may contain links to malicious content. DO NOT follow any links below

I think I found a workaround.

1. Play the video in a browser
2. While the video is playing (or paused), right-click on the video to get a pop-up menu
3. Select "Copy debug info"
4. That debug info contains "debug_videoId", eg "ABC123"
5. Profit: yt-dlp https://drive(dot)google.com/file/d/ABC123

nb. watch out how github automatically edited the URL.