rdblue
commented
9 years ago @tomwhite, can you take a look at this fix? Something strange is going on with Hadoop's Path. If there is a path in the file system with an escaped directory name, a%2Fb, you can access it if you instantiate the path from a String. But if you create a URI and then a Path from that, you end up with a double-escaped path.
fs.exists(new Path("/tmp/a%2Fb")); // => true
Path badPath = new Path(URI.create("/tmp/a%2Fb"));
fs.exists(badPath); // => false
badPath.toString(); // => "/tmp/a/b"This means that we can't call Path#getUri inside of getLocation. We have to get the path as a string and then create a URI from it. Similarly, we can't call new Path(location) without introducing a bug. This looks like a serious problem for the API because users will expect to be able to create paths from reported locations and URI is the right thing to return (I thought), but Path will mange that URI if there are escaped characters.
mkwhitacre
tomwhite
noslowerdna
This adds test for URIs with escaped characters and fixes handling in the FS PartitionView implementation. The implementation was accidentally unescaping URIs by using URI.create, which expects escaped characters, along with URI#getPath, which will remove escapes.
This also fixes the URI returned by getLocation. This previously used Path#getUri, but that method returns the Path's internal URI that is double-escaped. Returning a URI created from Path#toString is the correct behavior because it matches the Path passed in. Similarly, the merge and replace methods in the FS dataset implementation that rely on Paths created from those locations have been updated to correctly construct a Path from a URI by converting the URI to a String rather than setting the internal URI directly.