Investigate race condition if using filesystem and directory doesn't exist

[pnorman]

After cleaning out everything I think I saw an error where multiple processes tried to create the directory and failed