Reading file from Drive giving OSError

furkanyildiz commented 5 years ago

I can not read my files on drive. It's sometimes working but mostly giving OSError.

OSError: Can't read data (file read failed: time = Mon May 20 00:34:07 2019
, filename = '/content/drive/My Drive/train/trainX_file1', file descriptor = 83, errno = 5, error message = 'Input/output error', buf = 0xc71d3864, total read size = 42145, bytes this sub-read = 42145, bytes actually read = 18446744073709551615, offset = 119840768)

Also creating file giving the OSError.

OSError: Unable to create file (unable to open file: name = '/content/drive/My Drive/train/model.hdf5', errno = 5, error message = 'Input/output error', flags = 13, o_flags = 242)

You can check my nootebook to see the error. https://colab.research.google.com/drive/1MHJhYtR1PGyb5HKUY8-hrFPn-SBtOige

Note: https://research.google.com/colaboratory/faq.html#drive-timeout does not helped me.

nickagee commented 4 years ago

When I run the following command: !cp "/content/drive/My Drive/DL_Class/face-generation/data/processed-celeba-small.zip" /content/

I just received the below error: cp: error reading '/content/drive/My Drive/DL_Class/face-generation/problem_unittests.py': Input/output error

The file is a 2KB Python file.

edited formatting.

I just tried empying my Trash in my Google Drive and its still not working, saw this as a possible fix from a similar GH issue.

saransh09 commented 4 years ago

I was trying to follow the conversation but I am unable to catch up to it. I have Colab Pro, still facing this issue when I am using my Google Drive. Is there no work around this problem? Will I have to manually download and upload the files to the current runtime?

eyelovedata commented 4 years ago

Same problem, so frustrating. I also have a gsuite Google drive account and colab pro. Everything worked great until one day it just stopped. Has anyone else had this issue self resolve with time ?

saransh09 commented 4 years ago

Same problem, so frustrating. I also have a gsuite Google drive account and colab pro. Everything worked great until one day it just stopped. Has anyone else had this issue self resolve with time ?

Hey, even I started facing this problem right now. Probably Google's plan to force people on to GCP. Anyway! after spending hours trying to find my way out of it. https://medium.com/@acpanjan/download-google-drive-files-using-wget-3c2c025a8b99 This blog post is the best work around of it. Just download the required file on the present instance for now.

Note: You can still write data on your Google Drive. However, you won't be able to read from there. Sad, I know :/

Update: I was able to save the model, but apparently even that capability is gone now. Great! I feel cheated now. There's no point if I have to monitor the model training at every step. Really irritating.

DJW41023 commented 4 years ago

Google have just provided the update 'Colab is experiencing issues connecting to Drive, and we are actively investigating.' as a banner on all Colab workbooks

craigcitro commented 4 years ago

For anyone seeing new issues in the last 24 hours, see: https://github.com/googlecolab/colabtools/issues/1428

ghost commented 3 years ago

It fails even if i try to copy this file, the 2 unique solutions is to use google-drive-ocamlfuse or get public download link of the file and download it from your google colab. 189

leidix commented 3 years ago

I have the same problem and I'm even paying for Colab Pro. None of the solutions worked for me - very annoying

manastahir commented 3 years ago

For everyone, In case you are able to download the file by opening Google Drive in a web browser but not in the Colab environment: A common workaround that I recently discovered is using a chrome extension called CurlWget. The steps are as follows:

Download the CurlWget chrome extension.

Go to your Google drive, and click to download the file. As soon as the downloading starts, click on the CurlWget extension icon. It will show you a long command. You may now cancel the download that you started moments ago.

Copy the command to your Colab environment to download the file.

This one is the best work around for the problem. Thank you for sharing it!

jianmosier commented 3 years ago

I still have the OSError. It triggered by os.listdir(FILE_PATH). And the file only 500MB, but have 80000 JPG. And I have read it successful yesterday.

whcaa commented 9 months ago

I solved the problem.Place the shd_train.h5 file directly in the project without placing it in the hd_train.h5 folder and then in the project

deanp70 commented 5 months ago

Posting with permission from @cperry-goog - we're collaborating with the Colab team to provide DagsHub Storage as an alternative to GDrive that is more scalable and built for use with large datasets. It's an S3-compatible bucket that has much simpler access controls, and can be mounted easily.

It might help avoid the issues above - here's a link to an example notebook to try it out

We're looking for community feedback, so I'd love to get your input if it helps with the issue at hand.

(If you're curious, DagsHub is a platform for ML teams which is why we think Colab should have a storage solution suitable for ML workloads)

googlecolab / colabtools

Reading file from Drive giving OSError #559