Add function to make paths absolute, which is different from canonicalization

retep998 commented 5 years ago

Many users want to turn relative paths into absolute paths, but the only tool that libstd currently offers is canonicalize which is bad because it has to hit the filesystem which has several downsides:

It takes time to hit the filesystem.
The path has to actually exist.
It fails on certain types of drives such as RAM drives on Windows.
The path it creates is a \\?\ path which not all software can handle correctly and imposes requirements on further path manipulation that few users are even aware of.

Needing symbolic links actually resolved is an extremely rare use case, and is often misused to compare paths for equality when in reality you should be comparing file IDs due to things like hard links existing.

A new function (normalize or make_absolute or something, bikeshed away) should be added that will turn a relative path into an absolute path without touching the filesystem. On Windows this should either call GetFullPathNameW or do the pure Rust equivalent while on unixy platforms... something should happen, I have no idea.

Mark-Simulacrum commented 5 years ago

retep998 commented 5 years ago

One big question for the libs team to decide on is how to handle .. and . on linux. On Windows C:\foo\..\bar is equivalent to C:\bar and both paths will always point to the same thing. On linux however it is possible for /foo/../bar to not point to the same thing as /bar. Given a current directory of /foo and a relative path of ../bar, should normalize on linux return /foo/../bar or /bar?

Xanewok commented 5 years ago

Some prior art might be the Node path.normalize function which (it seems?) doesn't hit the FS and just tries to do the reasonable thing given path structure, e.g.:

For example, on POSIX:

path.normalize('/foo/bar//baz/asdf/quux/..');
// Returns: '/foo/bar/baz/asdf'

On Windows:

path.normalize('C:\\temp\\\\foo\\bar\\..\\');
// Returns: 'C:\\temp\\foo\\'

nayeemrmn commented 5 years ago

A function called normalize() will not be expected to convert to an absolute path, that name should be used for https://github.com/rust-lang/rfcs/issues/2208.

nayeemrmn commented 4 years ago

I guess fs::canonicalize() is like realpath -e and the requested feature is like realpath -ms on Linux.

I want to reiterate that fs::normalize() should not be used for this, but for rust-lang/rfcs#2208 which is different from both of these. It doesn't account for CWD and leaves relative paths relative, making it the proper equivalent of Node's path.normalize() referenced above.

Ciantic commented 3 years ago

I think this should be mentioned here too, @dylni has created a normpath crate to do pretty much this: https://crates.io/crates/normpath it seems to also call the GetFullPathNameW on Windows, and doesn't hit the disk.

dylni commented 3 years ago

Thanks @Ciantic!

To clarify, PathExt::normalize only doesn't touch the filesystem on Windows. On Unix, it needs to canonicalize the path to resolve it correctly. Canonical Unix paths don't have the same issues as verbatim paths on Windows.

PathExt::normalize_virtually from that crate can be used to normalize paths without touching the filesystem, but PathExt::normalize is usually what you want.

MouseProducedGames commented 3 years ago

"It fails on certain types of drives such as RAM drives on Windows."

Being able to compile on a RAM drive on Windows, would save a lot of SSD thrashing.

ChrisDenton commented 3 years ago

@MouseProducedGames Note that it only fails on improperly implemented RAM disk software (unfortunately this includes at least one rather popular one). Something well made, like Radeon RAMDisk, should work fine.

MouseProducedGames commented 3 years ago

@MouseProducedGames Note that it only fails on improperly implemented RAM disk software (unfortunately this includes at least one rather popular one). Something well made, like Radeon RAMDisk, should work fine.

That is good to know; and thanks for the recommendation.

andylizi commented 2 years ago

Should this be closed now that #91673 is merged?

gyk commented 1 year ago

It fails on certain types of drives such as RAM drives on Windows.

This appears to be fixed in the latest Windows 11 updates. I have tested calling canonicalize on (improperly implemented) ramdisk and network drive, and it no longer returns "Incorrect function". Could anyone verify it?

ChrisDenton commented 1 year ago

@gyk Do you have a version number? 10.0.22621 still fails last I checked (e.g. with ImDisk). Though I admit I haven't checked the latest Beta or Preview channels.

gyk commented 1 year ago

@ChrisDenton The OS builds are 22623.1325 (beta channel) and 22621.1105. The ramdisk software I tested was Ultra RAMDisk Lite (IIRC it didn't work before?), I just tried it again with ImDisk and yes, it still failed. And canonicalize now works on network drives mapped to QNAP or Synology NAS and I'm certain it used to return IO errors.

rust-lang / rust

Add function to make paths absolute, which is different from canonicalization #59117