search

fsspec / gcsfs

Pythonic file-system interface for Google Cloud Storage
http://gcsfs.readthedocs.io/en/latest/
BSD 3-Clause "New" or "Revised" License
342 stars 143 forks source link

GCSFileSystem doesn't properly handle cache_timeout = 0 #645

Open amyewang opened 4 days ago

amyewang commented 4 days ago

When using GCSFileSystem, I would expect passing in cache_timeout = 0 to result in no caching, since according to the docs: Set cache_timeout <= 0 for no caching, None for no cache expiration. Yet when cache_timeout = 0, self.cache_timeout is set to None and caching is being used.

When cache_timeout = 0, self.cache_timeout = cache_timeout or kwargs.pop("listings_expiry_time", None) is evaluating to None, since the value of 0 is falsy. Currently circumventing this by passing in cache_timeout = -1, but docs state that a value of 0 should be supported and indicate no caching.

martindurant commented 2 hours ago

OK, so the docstring and the functionality need to be brought into line - can you please make the PR? Ideally we should have the same behaviour between implementations, so the change might be in fsspec.dircache . I don't mind which sentinel value means "no caching", especially since we also have use_listings_cache. I suppose an expiry time of 0 could be understood as no caching, or as no expiry (i.e., use cache and never make stale).