Open d70-t opened 2 years ago
The following works:
with tempfile.TemporaryDirectory() as d:
m = fsspec.get_mapper(d, auto_mkdir=True)
m.setitems({"foo/bar": b"baz"})
So I'm not sure whether the mapper ought to be creating directories or not :)
Cool :+1: I didn't know about this option. But still, shouldn't be both of the two consistent (i.e. either both working or both raising?)
shouldn't be both of the two consistent
Yes, I'm just not sure which one is right. For object stores, it doesn't matter as there are no directories.
(or indeed, whether auto_mkdir should be True by default, which has also come up before)
I'd argue, if I use the mapper API, I want to treat the filesystem as if it were an object store, thus I'd expect an object-store like behavior.
But if auto_mkdir
is supposed to intentionally disable the creation of intermediate directories, then of course this still shouldn't happen...
Maybe auto_mkdir
should be something like None
by default, with the following semantics:
True
: always create intermediate directories if neededNone
: only create intermediate directories if an object-store like API is usedFalse
: never create intermediate directoriesThat sounds reasonable, but tricky to implement for the single backend, I think.
True.
When using a mapper on a local filesystem, it's possible to create nested subdirectory structures using
FSMap
as in:however, trying the same thing using
setitems
fails:I'd expect the latter to work just as well as the former, however, it raises:
This is likely because there's no
mkdirs
insetitems
, while there is one in__setitem__
. I don't know which place would be the best to put amkdirs
-like call when setting multiple items at once.