create unique temporary directories to prevent permission issues

ErikApption commented 2 weeks ago

Fixes #7172

This is a tentative fix for #7172 where ml.net fails when multiple users are using same directory. This fix checks if there is already a ml_dotnet if so, it will increase the number until the path is available.

[X] There's a descriptive title that will make sense to other developers some time from now.
[X] There's associated issues. All PR's should have issue(s) associated - unless a trivial self-evident change such as fixing a typo. You can use the format Fixes #nnnn in your description to cause GitHub to automatically close the issue(s) when your PR is merged.
[X] Your change description explains what the change does, why you chose your approach, and anything else that reviewers should know.
[X] You have included any necessary tests in the same PR.

michaelgsharp commented 2 days ago

so Path.GetRandomFileName() should make sure we are getting unique temp directories. What problems are you seeing in regards to this?

ErikApption commented 2 days ago

so Path.GetRandomFileName() should make sure we are getting unique temp directories. What problems are you seeing in regards to this?

Current code has this

string path = Path.Combine(Path.GetFullPath(tempPath), "ml_dotnet", Path.GetRandomFileName());

Issue is the directory name ml_dotnet, not the temp file name. The Path.Combine will create a subdirectory and a random file name inside that directory. The directory name is not unique. When one user processes creates /tmp/ml_dotnet/ for ML.net and a second user runs the same ML.net application at the same time, the second user cannot create /tmp/ml_dotnet/<temp file name> because permissions on /tmp/ml_dotnet/ are restricted to user 1. User 2 doesn't have write permissions (at least on default ubuntu but would be surprised this is different on other distros).

dotnet / machinelearning

create unique temporary directories to prevent permission issues #7173