iterative / PyDrive2

Google Drive API Python wrapper library. Maintained fork of PyDrive.
https://docs.iterative.ai/PyDrive2
Other
580 stars 69 forks source link

Inconsistency in the cache of `GDriveFileSystem` between item added during initialization and from ls #241

Closed simone-viozzi closed 10 months ago

simone-viozzi commented 1 year ago

Consider this minimal working example:

The mkdir method is from #222

def mkdir(fs: GDriveFileSystem, path, create_parents=True):
    """Create directory entry at path"""

    if fs.exists(path):
        raise FileExistsError(path)

    dst_id = fs._get_item_id(fs._parent(path), create=create_parents)
    basename = posixpath.basename(path.rstrip("/"))
    fs._gdrive_create_dir(dst_id, basename)

# create a GDriveFileSystem instance with a path that point to an empy folder
root = "root/tmp/"
fs = GDriveFileSystem(root, auth)

# dump the cache
print(json.dumps(fs._ids_cache, indent=4))
{
    "dirs": {
        "tmp": [
            "1BClMfgL7BMV61-5SdWAvc8njvkPgicyx"
        ]
    },
    "ids": {
        "1BClMfgL7BMV61-5SdWAvc8njvkPgicyx": "tmp"
    },
    "root_id": "1BClMfgL7BMV61-5SdWAvc8njvkPgicyx"
}

# create a folder
folder = posixpath.join(root, "folder")
mkdir(fs, folder)

# the cache did not change
print(json.dumps(fs._ids_cache, indent=4))
{
    "dirs": {
        "tmp": [
            "1BClMfgL7BMV61-5SdWAvc8njvkPgicyx"
        ]
    },
    "ids": {
        "1BClMfgL7BMV61-5SdWAvc8njvkPgicyx": "tmp"
    },
    "root_id": "1BClMfgL7BMV61-5SdWAvc8njvkPgicyx"
}

# use the ls method to get the content of the folder we just created
print(fs.ls(folder))
[]

# there is a new entry in the cache: "root/tmp/folder"
print(json.dumps(fs._ids_cache, indent=4))
{
    "dirs": {
        "tmp": [
            "1BClMfgL7BMV61-5SdWAvc8njvkPgicyx"
        ],
        "root/tmp/folder": [
            "1VKRRnGmZm_Dvmn0G8xQjqwQxbZ9vW0Hf"
        ]
    },
    "ids": {
        "1BClMfgL7BMV61-5SdWAvc8njvkPgicyx": "tmp",
        "1VKRRnGmZm_Dvmn0G8xQjqwQxbZ9vW0Hf": "root/tmp/folder"
    },
    "root_id": "1BClMfgL7BMV61-5SdWAvc8njvkPgicyx"
}

As we can see, there is an inconsistency between items added in the initialization phase of the cache and items added by ls:

Is this the expected behavior?

shcheklein commented 10 months ago

items added on the initialization of _ids_cache does not have the root/ prefix

This is not relevant anymore. Additional caching during init was removed.