insufficient support for character values

LeXofLeviafan commented 8 years ago

The character values (like \space) are parsed as strings, which is how they are normally represented is Python code anyway. However, there are two (in broad sense) issues with your current implementation (PyPI version).

The first issue is that a lot of character literals are simply unsupported; more specifically (comparing with default edn implementation in Clojure, clojure.edn/read-string):

for characters in ASCII printable range (codes 33-126, they're the same in Unicode), all of them should be representable the default way (\", !, \, etc.), but a quick check gives SyntaxError on 31 of them: !"#$%&'()*+,-./:;<=>?@[]^`{|}~
since edn specification doesn't seem to require reading unicode characters normally (though Clojure edn reader accepts them anyway... as well as whitespace), I've tried using allowed hex-code format (\uXXXX); however, when parsing "[\u007E]" (for example), loads() generated this output: [u'u', 7, Symbol(E)]

In general, Clojure edn parser seems to behave thus, upon encountering backslash:

keep reading&appending until a whitespace or a paren is encountered (by peeking in the stream)
if token is empty ("\"), read&append exactly once ("[\ 1 \]x]" is parsed as [\space 1 ] x])
discard backslash, process cases (hex-code, single Unicode char, known char name)

The second issue is that there is no way to emit character values with dumps(). Proposed solution would be providing a corresponding class for that specific purpose (named, for example, Char); alternatively, it might be used for reading as well, to keep type information (in this case, class probably should be extending str to keep backward-compatibility).

swaroopch commented 8 years ago

@LeXofLeviafan I'm not actively using EDN any more, so I don't know if I'll get around to this. In the meanwhile, pull requests are welcome.

bfontaine commented 6 years ago

To summary, there are three different issues here:

~~ASCII chars~~ → fixed by #45
~~Unicode chars~~ → fixed by #56
Char-like class

swaroopch commented 5 years ago

I believe this was fixed by the reporter @LeXofLeviafan in https://github.com/swaroopch/edn_format/pull/56

swaroopch commented 5 years ago

Oops, the third case remains open. So reopened.

swaroopch / edn_format

insufficient support for character values #31