sideshowbarker
commented
4 years ago I’ve raised a series of pull requests that fix all but one of the failures in the issue description.
Those pull requests are #30, #31, #32, #36, #37, #38, #39, and #40.
With those changes applied, the one remaining tokenization test from the html5lib-tests suite we’re still failing is this one:
https://github.com/html5lib/html5lib-tests/blob/master/tokenizer/domjs.test#L219-L223
"description":"lowercase endtags",
"initialStates":["RCDATA state", "RAWTEXT state", "Script data state"],
"lastStartTag":"xmp",
"input":"</XMP>",
"output":[["EndTag","xmp"]]And the following is the testharness failure message:
Failure
lowercase endtags
Input:
</XMP>
Expected tokens:
[["EndTag","xmp"]]
Actual tokens:
[["Character","</XMP>"]]I’ve not had time yet to investigate that one — but so far one thing I’ve noticed which seems odd about it is: Given that it has three states in its initialStates value, I’d expect that the harness would run it three times; but instead the harness apparently evaluates it only once. Also, the untokenized-</XMP>-as-character-data in the Actual tokens output seems to indicate that tokenization isn’t actually being run on the input at all.
We’re failing 30 tests cases in https://github.com/html5lib/html5lib-tests/tree/master/tokenizer/ (see below). At least 10 of them are related to handling of U+0000 NUL characters.
I don’t understand why the Java parser is failing these but the Firefox parser isn’t.