fsspec / ossfs

fsspec filesystem for Alibaba Cloud (Aliyun) Object Storage System (OSS)
Apache License 2.0
21 stars 11 forks source link

KeyError when DirCache is disabled #153

Open sutao opened 8 months ago

sutao commented 8 months ago

When I tried to disable DirCache by passing use_listings_cache=False or listings_expiry_time=0, any listing operation would raise a KeyError, this is due to fsspec's DirCache raising KeyError if no cached item is found.

The cause of the error is from ossfs/core.py line 183.

        try:
            self.dircache[norm_path] = self._get_object_info_list(
                bucket_name, prefix, delimiter, connect_timeout
            )
            return self.dircache[norm_path]  # <-- Raises KeyError if DirCache is disabled
        except oss2.exceptions.AccessDenied:
            return []

It should be changed to:

        try:
            obj_info_list = self._get_object_info_list(
                bucket_name, prefix, delimiter, connect_timeout
            )
            self.dircache[norm_path] = obj_info_list
            return obj_info_list
        except oss2.exceptions.AccessDenied:
            return []

There might be other places in the SDK that need fixing.