The testCreateWikiText103WithoutBpe test will always fail a precondition because it sets bpe to nil, while encodedFileName is also nil for that dataset. This test case is not active on Linux, but will run on macOS, where it crashes tests. As a result, I've removed this test case.
Additionally, the WordSegDataset used a hardcoded path to /dev/null when no filename is specified for two components. Direct access to /dev/null is blocked on macOS, and is problematic for Windows. I've replaced this with creation of an empty Data instance if the file name is missing.
The
testCreateWikiText103WithoutBpetest will always fail a precondition because it sets bpe to nil, whileencodedFileNameis also nil for that dataset. This test case is not active on Linux, but will run on macOS, where it crashes tests. As a result, I've removed this test case.Additionally, the WordSegDataset used a hardcoded path to /dev/null when no filename is specified for two components. Direct access to /dev/null is blocked on macOS, and is problematic for Windows. I've replaced this with creation of an empty Data instance if the file name is missing.
With these changes, all tests now pass on macOS.