Add support to UTF larger values (Emoticons;)

digitalbazaar / rdf-canonize

An implementation of the RDF Dataset Normalization Algorithm in JavaScript.

Other

22 stars 13 forks source link

Add support to UTF larger values (Emoticons;) #44

Closed krzysztofBakunSmartlogic closed 10 months ago

davidlehn commented 3 years ago

Tests are needed for this. Probably added to https://github.com/json-ld/rdf-dataset-canonicalization.
I think there may have been a reason we didn't do that simple change. Maybe something about it being tricky to get the encodings right in test files? Maybe some weird edge cases? I'm not sure. Just handling UTF-8 alone is easier. But I suppose emoticons are a critical use case. ;-) [<--- isn't ascii good enough?]

dlongley commented 2 years ago

@davidlehn -- any chance we can get a simple test for this and accept it?

krzysztofBakunSmartlogic commented 1 year ago

Hello, long time no see ;) I would like to add some tests to move forward with this pr but I have question. Acording to the readme and npm fetch-test-suite script we are using the https://github.com/w3c-ccg/rdf-dataset-canonicalization But the repo https://github.com/w3c-ccg/rdf-dataset-canonicalization is archived and moved to the https://github.com/w3c/rdf-canon/ ? Should I use the https://github.com/w3c/rdf-canon/ instead? thanks

davidlehn commented 1 year ago

A few months ago I did some work on escaping and added in a similar change. I also made some test cases that had unicode variations. I'll try to revive that work soon to get feedback on it. If you can wait a short while, you might not need to work on the test cases. And yes, the repo moved, and refs to it need to be updated.

The reason my changes got delayed is because they had an unfortunate performance hit, and I haven't added one basic mitigation for that yet.

davidlehn commented 1 year ago

A work in progress that includes this feature is here: https://github.com/digitalbazaar/rdf-canonize/pull/58.

davidlehn commented 10 months ago

Fixed in v4.0.0.