Using the system charset encoding can cause some tests to fail. I have changed these to convert explicitly using UTF8. The tests now pass. However, if I change charset to UTF_16, for example, the tests will fail - so there may be a bug somewhere. This is quite complicated as the return types are different:
The issue was arising when the Arbitrary[String] is generated with UTF-16 characters. Then calling getBytes() in Windows would convert any characters bigger than 0x00ff into a "?".
Using an explicit charset should have fixed this, but it only appears to work with UTF-8 (not UTF-16), which leads me to believe there may be another problem.
An alternative to all this would be to simply change the EncodingSample.text to an Array[Byte], instead of String, and do all validation on the byte arrays directly; base64 encoding shouldn't depend on the character encoding. However, as ByteVector.decodeBase64 returns a String, this would still need some fiddling.
Finally note that there is a java.util.Base64 implementation which may be handy for double-checking the codec is working properly.
Separately, I still have WebSocket.websocket-server failing. I haven't had a chance to look yet why. I can see it passes in Travis.
Using the system charset encoding can cause some tests to fail. I have changed these to convert explicitly using UTF8. The tests now pass. However, if I change
charsettoUTF_16, for example, the tests will fail - so there may be a bug somewhere. This is quite complicated as the return types are different:Stringinternal representation --> Array[Char] UTF-16String.getBytes()--> Array[Byte] system charsetString.getBytes(Charset)--> Array[Byte] specifiedCharsetByteVector.decodeBase64--> String == Array[Char], UTF-16ByteVector.decodeString(implicit Charset)--> Array[Byte] => String == Array[Char] UTF-16The issue was arising when the
Arbitrary[String]is generated with UTF-16 characters. Then calling getBytes() in Windows would convert any characters bigger than 0x00ff into a "?". Using an explicit charset should have fixed this, but it only appears to work with UTF-8 (not UTF-16), which leads me to believe there may be another problem.An alternative to all this would be to simply change the
EncodingSample.textto anArray[Byte], instead of String, and do all validation on the byte arrays directly; base64 encoding shouldn't depend on the character encoding. However, asByteVector.decodeBase64returns a String, this would still need some fiddling.Finally note that there is a
java.util.Base64implementation which may be handy for double-checking the codec is working properly.Separately, I still have
WebSocket.websocket-serverfailing. I haven't had a chance to look yet why. I can see it passes in Travis.