kowainik / tomland

🏝 Bidirectional TOML serialization
https://kowainik.github.io/posts/2019-01-14-tomland
Mozilla Public License 2.0
121 stars 39 forks source link

[Testing] Encode . decode test property #355

Open dariodsa opened 3 years ago

dariodsa commented 3 years ago

Encode . decode property-based testing is done on every data type in TOML, but there is an issue involving non-ascii characters (č, ž, щ, ё). The reason why that test fails for them is that those characters are converted into \Uxxxxxxxx and during the decode phase, they are not decoded back. The solution to this problem can be resolved in two ways:

  1. change decoding phase to turn \Uxxxxxxxx into regular characters
  2. change equal function that will say that č is equal to \U0000010d

I am a fan of the second approach because semantically speaking those two characters should be equal. I have already added some non-ascii characters to the test file, but I am not referring it to it anywhere in the code, as you can see down there. https://github.com/kowainik/tomland/blob/ee5b8facf71d79b482a80bce3d6de4dce3638d87/test/Test/Toml/Gen.hs#L304-L318