iterative / PyDrive2

Google Drive API Python wrapper library. Maintained fork of PyDrive.
https://docs.iterative.ai/PyDrive2
Other
570 stars 70 forks source link

GoogleDriveFile handle files with no fileSize #256

Closed NathanJGaul closed 1 year ago

NathanJGaul commented 1 year ago
fs = GDriveFileSystem(
    "root",
    google_auth=gauth,
)

for root, dnames, fnames in fs.walk(""):
    for dname in dnames:
        st.write(f"dir: {root}/{dname}")

    for fname in fnames:
        st.write(f"file: {root}/{fname}")

When using the GDriveFileSystem interface with the code above I am getting the following error:

Traceback (most recent call last):
  File "/home/nathanjgaul/development/doc-chat-gpt/venv/lib/python3.8/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 565, in _run_script
    exec(code, module.__dict__)
  File "/home/nathanjgaul/development/doc-chat-gpt/streamlit_app.py", line 121, in <module>
    main()
  File "/home/nathanjgaul/development/doc-chat-gpt/streamlit_app.py", line 112, in main
    for root, dnames, fnames in fs.walk("root"):
  File "/home/nathanjgaul/development/doc-chat-gpt/venv/lib/python3.8/site-packages/fsspec/spec.py", line 403, in walk
    listing = self.ls(path, detail=True, **kwargs)
  File "/home/nathanjgaul/development/doc-chat-gpt/venv/lib/python3.8/site-packages/pydrive2/fs/spec.py", line 457, in ls
    "size": int(item["fileSize"]),
  File "/home/nathanjgaul/development/doc-chat-gpt/venv/lib/python3.8/site-packages/pydrive2/files.py", line 227, in __getitem__
    return None
KeyError: KeyError('fileSize')

Taking a deeper look, it happens when interacting with files with no File Size property (see below).

image

I am able to handle this by changing GoogleDriveFile.__getitem__ to the following:

def __getitem__(self, key):
    """Overwrites manner of accessing Files resource.

    If this file instance is not uploaded and id is specified,
    it will try to look for metadata with Files.get().

    :param key: key of dictionary query.
    :type key: str.
    :returns: value of Files resource
    :raises: KeyError, FileNotUploadedError
    """
    try:
        return dict.__getitem__(self, key)
    except KeyError as e:
        if self.uploaded:
# fix here------------------
            if key == 'fileSize':
                return 0
#--------------------------
            raise KeyError(e)
        if self.get("id"):
            self.FetchMetadata()
            return dict.__getitem__(self, key)
        else:
            raise FileNotUploadedError()
shcheklein commented 1 year ago

@NathanJGaul 1.15.1 is released, please give it a try.