Open alexklwong opened 4 years ago
My solution is (same as above more or less)
- Click "Make A Copy" on google drive. It will create a copy of the file (either it's yours or other people's) to your drive.
- Set the permission to "Anyone with the link" as "Viewer".
- Now you can download as usual.
This works for single files or a folder. The problem is that the same error will occur if you are downloading a folder containing too many files. In that case, you can delete the folder and do the above process again, in this way, google will recognize them as different files so it doesn't err like "Too many users have viewed or downloaded".
The drawback of this is that it requires manual operation once the error occurs again.
IMO this is related to google drive not
gdown
so there is no point to do thosepip install
stuff
Google Driver now does not have "make a copy" feature, but "make a shortcut" instead, in this case, you can't have own copy of the target, which is very sad.
Updated to the latest version (gdown-4.7.1), tried using gdown "id...&confirm=t"
and nothing worked for me (47gb zip file):
Access denied with the following error:
Cannot retrieve the public link of the file. You may need to change
the permission to 'Anyone with the link', or have had many accesses.
You may still be able to access the file from the browser:
https://drive.google.com/uc?id=...&confirm=t
upd: worked for me after I changed permission type from 'editor' back to 'viewer'
upd2: same command failed on another machine. The 'cure' was just to wait for 10 mins and same command worked. idn why
Updated to the latest version (gdown-4.7.1), tried using
gdown "id...&confirm=t"
and nothing worked for me (47gb zip file):Access denied with the following error: Cannot retrieve the public link of the file. You may need to change the permission to 'Anyone with the link', or have had many accesses. You may still be able to access the file from the browser: https://drive.google.com/uc?id=...&confirm=t
upd: worked for me after I changed permission type from 'editor' back to 'viewer'
upd2: same command failed on another machine. The 'cure' was just to wait for 10 mins and same command worked. idn why
same issue here.
very strange that this is still an issue for almost entire nodejs time. Ive also had some encounters every once a while not being able to proceed just accept it. If it is indeed the cookie and its behaviour at warning etc. why is there no automated option still so
Still not solved 👎🏻
same here
It looks like google is trying to protect its bandwidth. After some time with downloading big files, it will be very slow and shutting down access later. Temporary solution I found : https://github.com/ericyd/gdrive-copy
Copy to your own folder, reshare it, coulde bypass this check
But the thing is that I'm able to download the file from the browser, but not with gdown. So, is there are possibility for gdown to simulate 'browser behavior' and pretend to be a browser for google?
For anyone who is still struggling with this problem, please refer to https://stackoverflow.com/questions/65312867/how-to-download-large-file-from-google-drive-from-terminal-gdown-doesnt-work or 解决命令行下载Google Drive大文件出现的问题.
Older version gdown also helps.
pip install gdown==4.6.0
I finally solved this issue by using 4.6.0 gdown to download files.
@e96031413 worked like a charm. thanks alot
pip install gdown==4.6.0
actually solved my issue. thanks
This PR #300 should help this issue in some regard. I also added FAQ to the README. https://github.com/wkentaro/gdown?tab=readme-ov-file#i-set-the-permission-anyone-with-link-but-still-cant-download
Releasing v5.0.0 soon.
When hitting large files that GDrive wants extra confirmation before it starts the download, e.g. https://drive.google.com/uc?id=1r6o0pSROcV1_VwT4oSjA2FBUSCWGuxLK, we currently hit this branch:
The relevant corresponding HTML looks like
<form id="download-form" action="https://drive.usercontent.google.com/download" method="get">
<input type="submit" id="uc-download-link" class="goog-inline-block jfk-button jfk-button-action" value="Download anyway"/>
<input type="hidden" name="id" value="1r6o0pSROcV1_VwT4oSjA2FBUSCWGuxLK">
<input type="hidden" name="confirm" value="t">
<input type="hidden" name="uuid" value="ce320bab-e9f0-4d0d-a9d8-8cede0161915">
</form>
The branch above just extracts https://drive.usercontent.google.com/download
from it and this later fails other checks as this doesn't provide a file.
What we should do here is to extract the other parts of the form as well and append them as query. So for this example https://drive.usercontent.google.com/download?id=1r6o0pSROcV1_VwT4oSjA2FBUSCWGuxLK&confirm=t&uuid=ce320bab-e9f0-4d0d-a9d8-8cede0161915
. As you can see this link also includes confirm=t
that multiple people in this thread have already mentioned. I'll send a PR to fix this.
@wkentaro see #308. Could you please have a look?
Hi everyone.
I've just merged #308 by @pmeier, which adds extra pattern to parse the url to download.
Since I've never seen that pattern, I suspect it is a regional issue (I'm accessing from Japan, @pmeier is accessing from Germany). But at least It should help people who have the same issue as him.
@wkentaro Hi, I observed that only downgrading to 4.6.0
helps, new version does not help :(
Can you please give me an example url?
I tried both
gdown --fuzzy 'https://drive.google.com/file/d/1NmGrEJ1uxrrBgf98UCU9SgpgsGEs3sLb/view?usp=drive_link'
and
gdown https://drive.google.com/uc?id=1NmGrEJ1uxrrBgf98UCU9SgpgsGEs3sLb
But it only works for me with gdown==4.6.0
@vanloc1808 It seems working for me with the latest.
Maybe the cookies file is not helping? If so remove it by rm ~/.cache/gdown/cookies.txt
@wkentaro it works for me, thank you!
Older version gdown also helps.
pip install gdown==4.6.0
I finally solved this issue by using 4.6.0 gdown to download files.
Restarting the terminal after the downgrading works for me!
I had a funny moment today. I got the same issue explained in the first message on the topic. The solution was to reinstall a several times version of gdown and finally install 5.1.0 ver to solve the problem. But this version had this issue when I started searching for the solution 😆
p.s. it is for colab
I had a funny moment today. I got the same issue explained in the first message on the topic. The solution was to reinstall a several times version of gdown and finally install 5.1.0 ver to solve the problem. But this version had this issue when I started searching for the solution 😆
p.s. it is for colab
Yeah this happened to me as well several times (with different versions). I'm not sure why, but changing the version (sometimes to a newer one, sometimes to an older one) seem to work, but always with a different version. Maybe it clears something and allows gdown to download again? I usually try versions 4.6.0, 5.1.0, or the latest (pip install --upgrade gdown). I also remove cookies.txt, and I end up (randomly) with a version that works. Hope this helps
I had a funny moment today. I got the same issue explained in the first message on the topic. The solution was to reinstall a several times version of gdown and finally install 5.1.0 ver to solve the problem. But this version had this issue when I started searching for the solution 😆
p.s. it is for colab
Weirdly this worked for me too. And I was using the default gdown (pip install gdown
) on colab using gdown --id ID
without a problem until just a few days ago. Thanks!
@wkentaro I'm still suffering this issue today. (gdown version 5.2.0)
I'm trying to download my own uploaded dataset (10G)
https://drive.google.com/uc?id=1vv73kAHiKb4KiL_oIH4DOWzUoaTeKzt_
And it raises the error
gdown https://drive.google.com/uc?id=1vv73kAHiKb4KiL_oIH4DOWzUoaTeKzt_
Failed to retrieve file url:
Too many users have viewed or downloaded this file recently. Please
try accessing the file again later. If the file you are trying to
access is particularly large or is shared with many people, it may
take up to 24 hours to be able to view or download the file. If you
still can't access a file after 24 hours, contact your domain
administrator.
You may still be able to access the file from the browser:
https://drive.google.com/uc?id=1vv73kAHiKb4KiL_oIH4DOWzUoaTeKzt_
but Gdown can't. Please check connections and **permissions.**
@JiaHeng-DLUT 's workaround works for me (https://stackoverflow.com/questions/65312867/how-to-download-large-file-from-google-drive-from-terminal-gdown-doesnt-work)
@ain-soph It works for me now, so looks like too many requests?
@JiaHeng-DLUT 's workaround works for me (https://stackoverflow.com/questions/65312867/how-to-download-large-file-from-google-drive-from-terminal-gdown-doesnt-work)
We can't do this because the scope of gdown is without logging in.
this is another workaround: https://github.com/wkentaro/gdown?tab=readme-ov-file#i-set-the-permission-anyone-with-link-but-i-still-cant-download
#!/bin/bash
# Prompt for access token
read -p "Enter your Google Drive access token: " ACCESS_TOKEN
# Define the input file containing URLs
INPUT_FILE="file_urls.txt"
# Define the output directory
OUTPUT_DIR="/mnt/harddrive/gdrive"
# Check if the input file exists
if [[ ! -f "$INPUT_FILE" ]]; then
echo "Input file '$INPUT_FILE' not found!"
exit 1
fi
# Check if the output directory exists
if [[ ! -d "$OUTPUT_DIR" ]]; then
echo "Output directory '$OUTPUT_DIR' does not exist!"
exit 1
fi
# Loop through each line in the input file
while IFS= read -r URL; do
# Extract the file ID from the URL
FILE_ID=$(echo "$URL" | grep -oP 'd/\K[^/]+')
# Check if FILE_ID was extracted successfully
if [[ -z "$FILE_ID" ]]; then
echo "Failed to extract file ID from URL: $URL"
continue
fi
# Get the file metadata to retrieve the original name
FILE_METADATA=$(curl -s -H "Authorization: Bearer $ACCESS_TOKEN" "https://www.googleapis.com/drive/v3/files/$FILE_ID?fields=name")
ORIGINAL_NAME=$(echo "$FILE_METADATA" | jq -r '.name')
# Check if the original name was retrieved successfully
if [[ -z "$ORIGINAL_NAME" ]]; then
echo "Failed to retrieve original name for file ID: $FILE_ID"
continue
fi
# Use curl to download the file to the specified output directory
OUTPUT_FILE="$OUTPUT_DIR/$ORIGINAL_NAME"
echo "Downloading file with ID: $FILE_ID as '$ORIGINAL_NAME' to '$OUTPUT_FILE'"
curl -H "Authorization: Bearer $ACCESS_TOKEN" -C - "https://www.googleapis.com/drive/v3/files/$FILE_ID?alt=media" -o "$OUTPUT_FILE"
# Check if the download was successful
if [[ $? -eq 0 ]]; then
echo "Downloaded: $OUTPUT_FILE"
else
echo "Failed to download file with ID: $FILE_ID"
fi
done < "$INPUT_FILE"
To anybody reading this
jq
from your distro package manager Going to OAuth 2.0 Playground https://developers.google.com/oauthplayground/ In the Select the Scope box, paste https://www.googleapis.com/auth/drive.readonly Click Authorize APIs and then Exchange authorization code for tokens Copy the Access token
This will save all the files that were mentioned in the file_urls.txt , just make there is only one url per line.
I want to download a 16GB zip file on Google Drive using:
gdown https://drive.google.com/uc?id=1InfIal4y7OBMGNUDeldEmDxtD0MrewY8
I have already set the permissions on the file to ``Anyone with the link''
But I get this error:
Permission denied: https://drive.google.com/uc?id=1InfIal4y7OBMGNUDeldEmDxtD0MrewY8 Maybe you need to change permission over 'Anyone with the link'?