search

shashi / FileTrees.jl

Parallel computing with a tree of files metaphor
http://shashi.biz/FileTrees.jl
Other
88 stars 6 forks source link

Issue with `mapsubtrees` #27

Closed xiaodaigh closed 4 years ago

xiaodaigh commented 4 years ago

I tried to do a mapsubtrees which after I should be able to do reducevalues but it is no allowing me. See MWE below 

mkdir("plsdel")
mkdir("plsdel/folder_a")
mkdir("plsdel/folder_b")

using DataFrames, FileTrees, CSV
a = DataFrame(a = 1:100)
b = DataFrame(a = 1:100)

CSV.write("plsdel/folder_a/a.csv", a)
CSV.write("plsdel/folder_b/a.csv", b)

ft = FileTree("plsdel")

ft2 = mapsubtrees(ft, r"folder") do file
    f = file |> FileTrees.path |> string
    CSV.read(joinpath("plsdel", f, "a.csv"))
end
reducevalues(vcat, ft2)
shashi commented 4 years ago 
reducevalues(vcat, ft2, dirs=true)

Arguably that should be default.