In Windows platform usage of ResourceId.resolve(..) method to resolve paths with glob expressions (containing multiple special characters like **, */* etc) is leading to the error java.nio.file.InvalidPathException: Illegal char <:> at index 2: /C:/.... This is because it internally uses Paths.get() which fails in Windows OS platform to resolve the paths correctly. Refer https://bugs.openjdk.org/browse/JDK-8197918 for details. Also, the documentation of this api ResourceId.resolve(..) does not talk about the support for glob expressions in general. Its best to avoid using this api for resolving paths containing multiple special characters.
As a fix, Instead of creating ResourceId to match for files, we can create string path specs with glob expressions and use this api FileSystems.match(..) to match the files, which documents about the support for glob expressions as well.
Unsolved: There are 2 warnings which are being reported when the pipelines are being executed in Windows platform, the details are documented here #869.
E2E test
Tested the Full run and Incremental Runs in Windows platform.
Tested the Full and Incremental runs for GCS File System on Unix platform.
TESTED:
Regression tests on Unix platform for Full and Incremental runs.
Description of what I changed
Fixes #843
In Windows platform usage of ResourceId.resolve(..) method to resolve paths with glob expressions (containing multiple special characters like
**
,*/*
etc) is leading to the errorjava.nio.file.InvalidPathException: Illegal char <:> at index 2: /C:/...
. This is because it internally usesPaths.get()
which fails in Windows OS platform to resolve the paths correctly. Refer https://bugs.openjdk.org/browse/JDK-8197918 for details. Also, the documentation of this api ResourceId.resolve(..) does not talk about the support for glob expressions in general. Its best to avoid using this api for resolving paths containing multiple special characters.As a fix, Instead of creating
ResourceId
to match for files, we can createstring path specs
with glob expressions and use this api FileSystems.match(..) to match the files, which documents about the support for glob expressions as well.Unsolved: There are
2
warnings which are being reported when the pipelines are being executed in Windows platform, the details are documented here #869.E2E test
Tested the
Full run
andIncremental Runs
in Windows platform. Tested theFull
andIncremental
runs for GCS File System on Unix platform.TESTED:
Regression tests on Unix platform for
Full
andIncremental
runs.Checklist: I completed these to help reviewers :)
[x] I have read and will follow the review process.
[x] I am familiar with Google Style Guides for the language I have coded in.
No? Please take some time and review Java and Python style guides.
[x] My IDE is configured to follow the Google code styles.
No? Unsure? -> configure your IDE.
[x] I have added tests to cover my changes. (If you refactored existing code that was well tested you do not have to add tests)
[x] I ran
mvn clean package
right before creating this pull request and added all formatting changes to my commit.[x] All new and existing tests passed.
[x] My pull request is based on the latest changes of the master branch.
No? Unsure? -> execute command
git pull --rebase upstream master