fix: Emoji support #53

[myConsciousness]

Hi amazing developers,

Issue #53 appears to have already been resolved in a previous pull request #123, but seems to be blocked due to CLA. So I have reopened a pull request on this issue instead.

Thank you.

[kevmoo]

@myConsciousness – could you drop the extra 0 that forces lines to change w/ no effect?

We also need tests here...

[natebosch]

I don't understand why we are dropping the range 0x10000 to 0x10FFFF in order to add this one.

I'm also curios about @kevmoo comment. Were we relying on this to have valid output before? Will we need some other logic to be spec compliant?

cc @jonasfj

[myConsciousness]

@kevmoo @natebosch - Thanks for your check!

I'm not familiar with the history of this library and would have liked to discuss the changes to the existing code made in #123 other than the emoji range.

I'm going to commit a patch with tests later :)

[jonasfj]

I think it might be wise to ask:

• (A) What does https://github.com/yaml/libyaml do?
• (B) What do other implementations do https://matrix.yaml.info/ ?
• (C) What does the spec say? maybe the relevant part is:

5.1. Character Set

To ensure readability, YAML streams use only the printable subset of the Unicode character set. The allowed character range explicitly excludes the C0 control block #x0-#x1F (except for TAB #x9, LF #xA, and CR #xD which are allowed), DEL #x7F, the C1 control block #x80-#x9F (except for NEL #x85 which is allowed), the surrogate block #xD800-#xDFFF, #xFFFE, and #xFFFF.

On input, a YAML processor must accept all Unicode characters except those explicitly excluded above.

[alestiago]

What's the status of this? (cc: @sigurdm , @jonasfj , @kevmoo)

[kevmoo]

Didn't we fix this somewhere else?

[tamcy]

@kevmoo Yes, fixed in #159. In master, not yet in release (is it ok to request one?).

[kevmoo]

Closing this out as it seems #159 fixed things.