shashi / FileTrees.jl

Parallel computing with a tree of files metaphor
http://shashi.biz/FileTrees.jl
Other
88 stars 6 forks source link

Issue with network paths on Windows systems #58

Closed abx78 closed 2 years ago

abx78 commented 2 years ago

This works fine

julia> mydir = FileTree("C:\\Temp\\TestData")
C:\Temp\TestData/
├─ file-179.csv
└─ file-216.csv

julia> a = FileTrees.load(mydir) do test
       DataFrame(CSV.File(path(test)))
       end
C:\Temp\TestData/
├─ file-179.csv (3×7 DataFrame)
└─ file-216.csv (4×7 DataFrame)

This does not :(

julia> mydir = FileTree("\\\\Network\\TestData")
\\Network\TestData/
├─ layer-179.csv
└─ layer-216.csv

julia> a = FileTrees.load(mydir) do test
       DataFrame(CSV.File(path(test)))
       end

This is the error:

ERROR: ArgumentError: \\Network\TestData cannot be parsed as FilePathsBase.AbstractPath
Stacktrace:
  [1] #parse#6
    @ C:\Users\me\.julia\packages\FilePathsBase\YFK4h\src\path.jl:74 [inlined]
  [2] parse
    @ C:\Users\me\.julia\packages\FilePathsBase\YFK4h\src\path.jl:73 [inlined]
  [3] Path
    @ C:\Users\me\.julia\packages\FilePathsBase\YFK4h\src\path.jl:22 [inlined]
  [4] Path(d::FileTree)
    @ FileTrees C:\Users\me\.julia\packages\FileTrees\eyEzJ\src\datastructure.jl:287
  [5] Path(d::File)
    @ FileTrees C:\Users\me\.julia\packages\FileTrees\eyEzJ\src\datastructure.jl:288
  [6] path(x::File)
    @ FileTrees C:\Users\me\.julia\packages\FileTrees\eyEzJ\src\datastructure.jl:291
  [7] (::var"#95#96")(test::File)
    @ Main .\REPL[98]:2
  [8] #77
    @ C:\Users\me\.julia\packages\FileTrees\eyEzJ\src\values.jl:37 [inlined]
  [9] #postwalk#20
    @ C:\Users\me\.julia\packages\FileTrees\eyEzJ\src\datastructure.jl:334 [inlined]
 [10] (::FileTrees.var"#18#19"{typeof(identity), FileTrees.var"#77#78"{Bool, Bool, var"#95#96"}})(c::File)
    @ FileTrees C:\Users\me\.julia\packages\FileTrees\eyEzJ\src\datastructure.jl:330
 [11] iterate
    @ .\generator.jl:47 [inlined]
 [12] _collect(c::Vector{Any}, itr::Base.Generator{Vector{Any}, FileTrees.var"#18#19"{typeof(identity), FileTrees.var"#77#78"{Bool, Bool, var"#95#96"}}}, #unused#::Base.EltypeUnknown, isz::Base.HasShape{1})
    @ Base .\array.jl:691
 [13] collect_similar(cont::Vector{Any}, itr::Base.Generator{Vector{Any}, FileTrees.var"#18#19"{typeof(identity), FileTrees.var"#77#78"{Bool, Bool, var"#95#96"}}})
    @ Base .\array.jl:606
 [14] map(f::Function, A::Vector{Any})
    @ Base .\abstractarray.jl:2294
 [15] postwalk(f::FileTrees.var"#77#78"{Bool, Bool, var"#95#96"}, t::FileTree; collect_children::typeof(identity))
    @ FileTrees C:\Users\me\.julia\packages\FileTrees\eyEzJ\src\datastructure.jl:330
 [16] postwalk(f::Function, t::FileTree)
    @ FileTrees C:\Users\me\.julia\packages\FileTrees\eyEzJ\src\datastructure.jl:330
 [17] #load#76
    @ C:\Users\me\.julia\packages\FileTrees\eyEzJ\src\values.jl:40 [inlined]
 [18] load(f::Function, t::FileTree)
    @ FileTrees C:\Users\me\.julia\packages\FileTrees\eyEzJ\src\values.jl:34
 [19] top-level scope
    @ REPL[98]:1

Some more information:

julia> versioninfo()
Julia Version 1.6.2
Commit 1b93d53fc4 (2021-07-14 15:36 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Xeon(R) Gold 6242 CPU @ 2.80GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, skylake-avx512)

Thanks for any help you can provide!

A.

abx78 commented 2 years ago

After further investigation I realized the issue is caused by the FilePathsBase package, at the moment it does not support UNC paths. https://github.com/rofinn/FilePathsBase.jl/blob/master/src/windows.jl#L42

this is a PR open since 2018 to fix the issue. https://github.com/rofinn/FilePathsBase.jl/pull/12

I am closing this issue, I will refer to that repository.