Tokenize Multi-character Tokens - Githubissues

andrewclarkson / yaml.rs

MIT License

2 stars 0 forks source link

Tokenize Multi-character Tokens #7

Open andrewclarkson opened 9 years ago

andrewclarkson commented 9 years ago

Correctly tokenize multi-character tokens like:

the document start token (---)
the document end token (...)

(aka ns-ns-document-start and ns-ns-document-end in the spec)

Necessary for #2