PradyumnaKrishna / Colab-Hacks

Simple Hacks for Google Colaboratory to boost your productivity and help you to perform daily tasks.
MIT License
890 stars 916 forks source link

Google Drive Mount Issue #70

Closed zaksleto closed 2 years ago

zaksleto commented 2 years ago

Traceback (most recent call last): File "/content/mount.py", line 7, in drive.mount('/home/user/drive') File "/usr/local/lib/python3.7/dist-packages/google/colab/drive.py", line 113, in mount ephemeral=ephemeral) File "/usr/local/lib/python3.7/dist-packages/google/colab/drive.py", line 296, in _mount raise ValueError('mount failed: invalid oauth code') ValueError: mount failed: invalid oauth code

PriyanshuNaredi commented 2 years ago

Same Issue With Me

KaiHLi commented 2 years ago

Same Issue here

PradyumnaKrishna commented 2 years ago

Temporary Solution: https://github.com/googlecolab/colabtools/issues/2563#issuecomment-1018186027

temp192000 commented 2 years ago

Temporary Solution: googlecolab/colabtools#2563 (comment)

Hey, Have you tried this solution? I'd tried it but, didn't work for me though.

ITtest1 commented 2 years ago

临时解决方案:googlecolab/colabtools#2563(评论)

You fixed it before, can you fix it again.

PradyumnaKrishna commented 2 years ago

Problem

This repository highly depend on Google Colaboratory and unfortunately, due to internal changes current method has stopped working.

Solution

You can use the code below to mount Google Drive for now, I will commit this soon or you can create a PR.

! apt-get install -y -qq software-properties-common python-software-properties module-init-tools
! add-apt-repository -y ppa:alessandro-strada/ppa 2>&1 > /dev/null
! apt-get update -qq 2>&1 > /dev/null
! apt-get -y install -qq google-drive-ocamlfuse fuse

import getpass
from google.colab import auth
from oauth2client.client import GoogleCredentials

try:
    mountpoint = f"/home/{username}/drive"
except NameError:
    print("'username' variable not found, mounting at `/content/drive`")
    mountpoint = '/content/drive'

auth.authenticate_user()
creds = GoogleCredentials.get_application_default()
! google-drive-ocamlfuse -headless -id={creds.client_id} -secret={creds.client_secret} < /dev/null 2>&1 | grep URL
vcode = getpass.getpass()
! echo {vcode} | google-drive-ocamlfuse -headless -id={creds.client_id} -secret={creds.client_secret}

! mkdir -p $mountpoint
! google-drive-ocamlfuse -o allow_other $mountpoint

Notes

Next Steps

We have a solution but look for other way around to solve this issue because it requires some dependencies and twice authentication.

If anyone is willing to contribute, you can commit the code in notebook, removing old one and open a Pull Request to close this issue. Tell me in the comment and I will assign you.

reachrishav commented 2 years ago

I am unable to access shared drives using the above solution. Is there any method to access the same?

ITtest1 commented 2 years ago

Mounted, but not accessible. It is empty.

PradyumnaKrishna commented 2 years ago

I am unable to access shared drives using the above solution. Is there any method to access the same?

@reachrishav, you can't access team drive or any shortcut linked to restricted access using google-drive-ocamlfuse, that can be mounted using label read in https://github.com/astrada/google-drive-ocamlfuse/wiki/Team-Drives


@ITtest1 I can assure you its working as I can list files in drive.

reachrishav commented 2 years ago

I am unable to access shared drives using the above solution. Is there any method to access the same?

@reachrishav, you can't access team drive or any shortcut linked to restricted access using google-drive-ocamlfuse, that can be mounted using label read in https://github.com/astrada/google-drive-ocamlfuse/wiki/Team-Drives

@ITtest1 I can assure you its working as I can list files in drive.

@PradyumnaKrishna How do I use the above mentioned workaround with your solution? I cant seem to find the gdfuse config file that is used in astrada's workaround.

PradyumnaKrishna commented 2 years ago

@PradyumnaKrishna How do I use the above mentioned workaround with your solution? I cant seem to find the gdfuse config file that is used in astrada's workaround.

let me give you an example, you need to replace the command with

- ! echo {vcode} | google-drive-ocamlfuse -headless -id={creds.client_id} -secret={creds.client_secret}
+ ! echo {vcode} | google-drive-ocamlfuse -headless -id={creds.client_id} -secret={creds.client_secret} -label <label>

Now a config will get generated, according to colab the path will be /root/.gdfuse/<label>/config look for the team_drive_id in config and paste it. at last, mount command will change to

! google-drive-ocamlfuse -label <label> -o allow_other $mountpoint

When I get time I will play around google-drive-ocamlfuse and build some more functionality. There are not many alternatives present.

Feel free to test the code or solution provided and contribute by creating a Pull Request.

Chetan-Goyal commented 2 years ago

I want to work on this issue.

reachrishav commented 2 years ago

It works! Thank you 👍 Looking forward to a simple and easy fix soon

temp192000 commented 2 years ago

It works! Thank you 👍 Looking forward to a simple and easy fix soon

Hey, I too tried ocamlfuse workaround. But, neither the existing files in the mounted drives aren't visible nor am I able to do Read/Write operations on the mounted drives. Is it the same with you?

Edit: @PradyumnaKrishna @Chetan-Goyal, anybody experienced this?

PradyumnaKrishna commented 2 years ago

@temp192000 @ITtest1 and other participants

Working on mounting team drives functionality #74, mostly part has been built. Some optimisations and error handing remains as you can get error sometime. If anyone want to test use notebook present at multiple-drives branch.

Notebook: https://github.com/PradyumnaKrishna/Colab-Hacks/blob/multiple-drives/Colab%20RDP/Colab%20RDP.ipynb Colab Link: https://colab.research.google.com/github/PradyumnaKrishna/Colab-Hacks/blob/multiple-drives/Colab%20RDP/Colab%20RDP.ipynb Pull Request #75 Mount shared drive and multiple drives

tfragment commented 2 years ago

It appears that the new method has some type of restriction on de/compressing files directly from the drives; the terminal displaying the message is simply stuck (the files may be too large) and yes, it's able to de/compress, but it is slow and occasionally it simply doesn't write any temp files when compressing (the size of temp files doesn't change).

Previously, the process was quick; however, this is no longer the case. This is so sad. lol. (This is not an accurate benchmark, it's just the feeling and what I was remembering.)

Now: Compressing 1GB+ took in the range of 20–40 min. Past: Compressing 1GB+ took in the range of 5–30 minutes, even while downloading and writing inside the drives were still going on. message of compressing on the terminal was not stuck or anything.

Command: cd "/home/user/drives/Drives A/A/"; zip -r -9 -s 500m "/home/user/drives/Drives A/B/__zip/Folder/Compressed.zip" "Folder"

tfragment commented 2 years ago

This what happens if downloading/writing is going on or de/compressing with multiple instances inside the drives. Compressing ended with an error when i was compressing split files with multiple instances, it wasn't like this before. :( (ex: Compressing Folder A and Compressing Folder B in one go): zip I/O error: Bad address zip error: Output file write failure (Could not write split)

PradyumnaKrishna commented 2 years ago

@tfragment, Experiment yourself with the config of google-drive-ocamlfuse and mount it https://github.com/astrada/google-drive-ocamlfuse/wiki/Configuration

tfragment commented 2 years ago

@PradyumnaKrishna Will try, Tysm!