search

googlecolab / colabtools

Python libraries for Google Colaboratory
Apache License 2.0
2.18k stars 709 forks source link

Mounting Google drive timeout #494

Closed Behtash-BehinAein closed 5 years ago

Behtash-BehinAein commented 5 years ago

Bug report for Colab: http://colab.research.google.com/.

For questions about colab usage, please use stackoverflow.

colaboratory-team commented 5 years ago

The notebook linked above shows cell execution completed in 37s and shows no timeout error(?). That said, your text about the folder that the colab notebook resides in belies a potential misunderstanding of the FAQ entry: it is irrelevant which folder the notebook is stored in. What matters is the direct-child count of the root of your Google Drive (for the mount() call) and of any directory you attempt to do I/O in (after the mount() succeeds). Please ensure that the root of your Google Drive doesn't have too many direct children.

Behtash-BehinAein commented 5 years ago

Thanks for your prompt reply.

Sorry for the confusion regarding the root of Google drive. I originally had many files in there when I first encountered the problem. Then I removed many files such that I was left with less than 20 direct-child count in the root (https://drive.google.com/drive/u/1/my-drive) of Google drive. All the other unsuccessful tests I performed was after that.

It is strange there was no time out error in the link. I think it disappeared. Here is a new link. I also copy pasted the error below. https://colab.research.google.com/drive/1AOONvdXgMlDuSG7KNlQ3oGTdBh1cqAgP

Enter your authorization code: ··········

ValueError Traceback (most recent call last)

in () 1 from google.colab import drive ----> 2 drive.mount('/content/drive') /usr/local/lib/python3.6/dist-packages/google/colab/drive.py in mount(mountpoint, force_remount, timeout_ms) 179 ': timeout during initial read of root folder; for more info: ' 180 'https://research.google.com/colaboratory/faq.html#drive-timeout') --> 181 raise ValueError('mount failed' + extra_reason) 182 elif case == 2: 183 # Not already authorized, so do the authorization dance. ValueError: mount failed: timeout during initial read of root folder; for more info: https://research.google.com/colaboratory/faq.html#drive-timeout
Behtash-BehinAein commented 5 years ago

Thanks for your prompt reply.

Sorry for the confusion regarding the root of Google drive. I originally had many files in there when I first encountered the problem. Then I removed many files such that I was left with less than 20 direct-child count in the root (https://drive.google.com/drive/u/1/my-drive) of Google drive. All the other unsuccessful tests I performed was after that. So at the time of reporting this issue, I had made sure there are few files in the root directory.

Link with error: It is strange there was no time out error in the link. I think it disappeared. Here is a new link. I've also copy pasted the error below.

https://colab.research.google.com/drive/1AOONvdXgMlDuSG7KNlQ3oGTdBh1cqAgP Enter your authorization code: ··········

ValueError Traceback (most recent call last) in () 1 from google.colab import drive ----> 2 drive.mount('/content/drive')

/usr/local/lib/python3.6/dist-packages/google/colab/drive.py in mount(mountpoint, force_remount, timeout_ms) 179 ': timeout during initial read of root folder; for more info: ' 180 'https://research.google.com/colaboratory/faq.html#drive-timeout') --> 181 raise ValueError('mount failed' + extra_reason) 182 elif case == 2: 183 # Not already authorized, so do the authorization dance.

ValueError: mount failed: timeout during initial read of root folder; for more info: https://research.google.com/colaboratory/faq.html#drive-timeout

On Sat, Mar 23, 2019 at 10:29 AM colaboratory-team notifications@github.com wrote:

The notebook linked above shows cell execution completed in 37s and shows no timeout error(?). That said, your text about the folder that the colab notebook resides in belies a potential misunderstanding of the FAQ entry https://research.google.com/colaboratory/faq.html#drive-timeout: it is irrelevant which folder the notebook is stored in. What matters is the direct-child count of the root of your Google Drive (for the mount() call) and of any directory you attempt to do I/O in (after the mount() succeeds). Please ensure that the root of your Google Drive doesn't have too many direct children.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/googlecolab/colabtools/issues/494#issuecomment-475888963, or mute the thread https://github.com/notifications/unsubscribe-auth/AYykfNp9ewcbToqxfrl-5MNPizewf0dyks5vZmSJgaJpZM4cE_5a .

colaboratory-team commented 5 years ago

Thanks for the info, @Behtash-BehinAein. Are you able to repro after choosing "Reset all runtimes" from the Runtime menu? If yes, can you "Reset all runtimes" again and try with a different Google account (if you have one) for Google Drive, but the same account for colab?

Behtash-BehinAein commented 5 years ago

Thanks for your guidance. It is much appreciated.

Yes. I can reproduce after "Reset all runtimes".

Here is what I did based on your instructions:

(1) Emailed a link of the colab notebook (that has error) from the current google account to my "other google account". (2) "Rest all runtimes" after opening the shared colab notebook (that has error) in the "other google account" (3) Successfully mounted google drive. Obviously this just mounted the drive of my "other google account".

Below is a snapshot of "My Drive" (I believe this is the root) in the account that has the error. It looks like there are only a few direct-childs here. There used to be 1000's of files in here, but I deleted them a week ago. Is it possible that I don't see the files but they are actually here ?

[image: image.png]

On Mon, Mar 25, 2019 at 11:02 AM colaboratory-team notifications@github.com wrote:

Thanks for the info, @Behtash-BehinAein https://github.com/Behtash-BehinAein. Are you able to repro after choosing "Reset all runtimes" from the Runtime menu? If yes, can you "Reset all runtimes" again and try with a different Google account (if you have one) for Google Drive, but the same account for colab?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/googlecolab/colabtools/issues/494#issuecomment-476313528, or mute the thread https://github.com/notifications/unsubscribe-auth/AYykfOIKaPGW9C_FF9DGxhdr0UsqyKgFks5vaQ89gaJpZM4cE_5a .

colaboratory-team commented 5 years ago

The image didn't come through. Up-thread you mentioned https://drive.google.com/drive/u/1/my-drive which implies the use of multi-login (the 1 in the URL). Can you confirm that your Drive root has few files for the account in question in an incognito chrome window signed in only to that account?

Behtash-BehinAein commented 5 years ago

Sure thing.

I signed into the problematic account through an incognito window. Now the address is https://drive.google.com/drive/my-drive.

I'm pasting and attaching the snapshot of what I see in the root. There is nothing I can see in there more than what we can see in the attachement.

By the way, I opened a new colab notebook in the incognitow chrome, reset all runtime and tried mounting google drive. Same time-out issue reproduced.

Regards, Behtash

[image: image.png]

On Mon, Mar 25, 2019 at 12:40 PM colaboratory-team notifications@github.com wrote:

The image didn't come through. Up-thread you mentioned https://drive.google.com/drive/u/1/my-drive which implies the use of multi-login (the 1 in the URL). Can you confirm that your Drive root has few files for the account in question in an incognito chrome window signed in only to that account?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/googlecolab/colabtools/issues/494#issuecomment-476347856, or mute the thread https://github.com/notifications/unsubscribe-auth/AYykfEUzbBZDpX7AO_jAPuS1wPzIHuBeks5vaSYmgaJpZM4cE_5a .

colaboratory-team commented 5 years ago

Image again failed to attach. Maybe try emptying Drive's trash?

Behtash-BehinAein commented 5 years ago

Sorry for the confusion. There is an actuall attachment to this email for the image. I didn't paste it this time in the body of email. (I think that is the one not showing up)

Emptying the trash would be a nightmare. It will take me days as I can only empty 10's to a couple of 100's of files at a time. Is there is script or a recursive way I can do it? Otherwise, I have to sit there and do it manually.

Regards, Behtash

On Mon, Mar 25, 2019 at 1:15 PM colaboratory-team notifications@github.com wrote:

Image again failed to attach. Maybe try emptying Drive's trash?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/googlecolab/colabtools/issues/494#issuecomment-476359441, or mute the thread https://github.com/notifications/unsubscribe-auth/AYykfO4-Os_LZITPlAYGc2Jz3ymqSpKvks5vaS5HgaJpZM4cE_5a .

colaboratory-team commented 5 years ago

I suspect you'll have to attach images on the github view of the bug, as they are (still) being stripped from email replies. You should be able to empty the trash directly at https://drive.google.com/drive/trash by clicking the down-arrow next to the Trash heading and then "Empty trash": empty-trash

Behtash-BehinAein commented 5 years ago

Got it. Here is the Github link to the snapshot : https://github.com/Behtash-BehinAein/Handy_Programs/blob/master/Snapshot%20of%20My%20Drive.JPG

I'm afraid the mehtod you kindly shared with me only deletes one page at a time, but it does not empty whole trash. I've actually researched this a lot in the past. This seems to be actualy the behavior of Google drive. Others have faced similar issues regarding emptying the trash with many files in it. There are various back and forths regarding this online. At the end Google folks don't seem to offer any method that can empty the whole trash.

Regards, Behtash

On Mon, Mar 25, 2019 at 2:14 PM colaboratory-team notifications@github.com wrote:

I suspect you'll have to attach images on the github view of the bug, as they are (still) being stripped from email replies. You should be able to empty the trash directly at https://drive.google.com/drive/trash by clicking the down-arrow next to the Trash heading and then "Empty trash": [image: empty-trash] https://user-images.githubusercontent.com/38081706/54954596-3dff8780-4f08-11e9-9421-5650c86022e8.png

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/googlecolab/colabtools/issues/494#issuecomment-476380037, or mute the thread https://github.com/notifications/unsubscribe-auth/AYykfBRxl1GYBF7QvxKEDhqTdraYr1t9ks5vaTwlgaJpZM4cE_5a .

Behtash-BehinAein commented 5 years ago

UPDATE:

I started emptying trash manually after you kindly mentioned doing so

to me. Somehow selecting all (ctrl+A) in the trash folder would select about a 1000 files at a time. It used to be 10's to a couple of 100's. I kept doing this till the trash folder was emptied.

Google Drive now mounts just fine. So the issue has been resolved. Thanks for your help and support.

Regards, Behtash

On Mon, Mar 25, 2019 at 2:53 PM Behtash Behin-Aein < behtash.behin.aein@gmail.com> wrote:

Got it. Here is the Github link to the snapshot : https://github.com/Behtash-BehinAein/Handy_Programs/blob/master/Snapshot%20of%20My%20Drive.JPG

I'm afraid the mehtod you kindly shared with me only deletes one page at a time, but it does not empty whole trash. I've actually researched this a lot in the past. This seems to be actualy the behavior of Google drive. Others have faced similar issues regarding emptying the trash with many files in it. There are various back and forths regarding this online. At the end Google folks don't seem to offer any method that can empty the whole trash.

Regards, Behtash

On Mon, Mar 25, 2019 at 2:14 PM colaboratory-team < notifications@github.com> wrote:

I suspect you'll have to attach images on the github view of the bug, as they are (still) being stripped from email replies. You should be able to empty the trash directly at https://drive.google.com/drive/trash by clicking the down-arrow next to the Trash heading and then "Empty trash": [image: empty-trash] https://user-images.githubusercontent.com/38081706/54954596-3dff8780-4f08-11e9-9421-5650c86022e8.png

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/googlecolab/colabtools/issues/494#issuecomment-476380037, or mute the thread https://github.com/notifications/unsubscribe-auth/AYykfBRxl1GYBF7QvxKEDhqTdraYr1t9ks5vaTwlgaJpZM4cE_5a .

colaboratory-team commented 5 years ago

Thanks so much for bearing with this process and reporting back the resolution!

colaboratory-team commented 5 years ago

(we'll update the FAQ entry to mention files in the Trash; thanks again!)

Behtash-BehinAein commented 5 years ago

The feeling is mutual. Also, your support motivated me to push through and try. And thanks for updating FAQ's. Makes the experience of all of us better.

By the way, not sure how easy it is, but being able to empty trash that contains 1000's files via one-click would be awesome ! 1000's of files can regularly get generated in seconds so gradually emptying trash is not an option.

Regards, Behtash

On Mon, Mar 25, 2019 at 3:40 PM colaboratory-team notifications@github.com wrote:

(we'll update the FAQ entry to mention files in the Trash; thanks again!)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/googlecolab/colabtools/issues/494#issuecomment-476405294, or mute the thread https://github.com/notifications/unsubscribe-auth/AYykfPUK0p_bkRvj4XUAJtr86Mxd0Y-tks5vaVB6gaJpZM4cE_5a .

lisatsinis commented 5 years ago

i've had a similar issue except the issue still is not resolved after taking the steps listed here.

ValueError Traceback (most recent call last)

in () 1 from google.colab import drive ----> 2 drive.mount('/content/drive') /usr/local/lib/python3.6/dist-packages/google/colab/drive.py in mount(mountpoint, force_remount, timeout_ms) 178 ': timeout during initial read of root folder; for more info: ' 179 'https://research.google.com/colaboratory/faq.html#drive-timeout') --> 180 raise ValueError('mount failed' + extra_reason) 181 elif case == 2: 182 # Not already authorized, so do the authorization dance. ValueError: mount failed
colaboratory-team commented 5 years ago

@lisatsinis please open a new issue with a minimal notebook showing the issue reproducing immediately after running "Reset all runtimes".

emarcari commented 4 years ago

I know that this thread is closed, but I have a question about this: If I have multiple access (different accounts) to a shared folder, it can be the reason of drive's timeout (and, as consequence, the collab error when execution and imediate reset?)

eysilka commented 4 years ago

Enter "the link you are provided after click on the link on the current notebook you are running" instead of "password" then it will be OK, Dear Sir.

wanfuse123 commented 4 years ago

I have found the reason why one cant mount ones own google drive for these things is because of a race condition with google . First it was suggested that changing the mount location from /content/gdrive to /content/something else but this didnt fix it. What I ended up doing was copying manually the files that are copied to google drive, then installing the google drive desktop application I would then in windows 10 go to the folder which is now located on google drive and disable file permissions inheritance and then manually putting full control rights on the folder to the users group and to authenticated users group. This seems to have fixed this for me. Other times I have noticed with these colabs (not this one in particular but some of the components used like the trained models are missing from the repository (as if they had been removed) Only solution for this is to look around for other sources of these files. This includes scurrying through google search engine and also looking at the git checkout level to find branches besides master and also looking for projects that cloned the project on github to see if they still include the files. Hope this helps!