It removes the pre-caching logic of root/00, root/01, .... root/ff DVC cache specific directories. It existed to optimize number of API calls, specifically when we use exists calls on the DVC transfer side and we have to do 2x calls in some simple scenarios. It's slower and expensive.
With this PR cache is becoming more general (e.g. we cache all directories we create). It makes sense for fsspec. To make it clear - the whole fsspec implementation in general case is still very broken and it'll be addressed step by step eventually.
The optimization can be turned back on with ad additional "pre_cache_path" argument. But let's do it after we fix the basic GDrive scenario with this one.
To mitigate https://github.com/iterative/dvc-gdrive/issues/38
It removes the pre-caching logic of
root/00,root/01, ....root/ffDVC cache specific directories. It existed to optimize number of API calls, specifically when we useexistscalls on the DVC transfer side and we have to do 2x calls in some simple scenarios. It's slower and expensive.With this PR cache is becoming more general (e.g. we cache all directories we create). It makes sense for
fsspec. To make it clear - the wholefsspecimplementation in general case is still very broken and it'll be addressed step by step eventually.The optimization can be turned back on with ad additional "pre_cache_path" argument. But let's do it after we fix the basic GDrive scenario with this one.