Closed sophiakoulen closed 1 year ago
This line should be modified to include a || *str == '#'
, and like you pointed out, while we're at it we can remove the redundant CRFL check as it's triggered on a \n
and \r
is white space that will get trimmed.
// line 30 in tokenizer.c
if (*str == '\0' || *str == '\n' || (*str == '\r' && *str + 1 == '\n'))
Stop tokenizing at hashtag