Helper functions for recursively getting files and subdirectories

[MartinSStewart]

I noticed that while there is Script.Directory.listFiles and Script.Directory.listSubdirectories they don't list any items in subdirectories.

I think it would be useful to have something like this too (and something similar for recursively listing all subdirectories):

listAllFiles : Script.Directory.Directory permissions -> Script String (List (Script.File.File permissions))
listAllFiles directory =
    Script.map2 (++)
        (Script.Directory.listFiles directory)
        (Script.Directory.listSubdirs directory
            |> Script.thenWith (List.map listAllFiles >> Script.sequence >> Script.map List.concat)
        )

This implementation doesn't bother handling edge cases like "What happens if we fail to read a file/directory after it gets listed in a subdirectory?" and "What happens if there's a symbolic link that to directory higher up, trapping us in a loop?" I'm not sure what should be done in these cases.

[ianmackenzie]

What sorts of operations would you then want to do with the returned files? I'm guessing that you'd often want to know what subdirectory each file is in, but there aren't currently very good ways of doing that. There's File.path which returns a String, but I'd love to remove even that since it can mean that a script restricted to executing within a certain directory can 'see' some things outside that directory (such as the current user's username, assuming the directory is somewhere within their home directory).

Recursing manually doesn't have those concerns, and gives you more control since you can start from a known parent directory and keep track of directory subdirectory names as you recurse into subdirectories (as is done for example here).

That said, I can see some potential good use cases for a listAllFiles or listFilesRecursively - maybe you want to run some automated refactoring on a bunch of files in place or something, and you don't really care where they are. I'm struggling a bit more with listSubdirectoriesRecursively...

[MartinSStewart]

My use case involves reading in all the Elm modules from a package directory. So the absolute path isn't a concern for me, I just need the text contents from each file that ends with .elm (which I get by then filtering by Script.File.name).