Open arp242 opened 6 years ago
Since termIndexes is a []int and some "raw" byte indexes are accessed (e.g. str[t+1] == byte(prefix)), multibyte prefixes won't work. This:
termIndexes
[]int
str[t+1] == byte(prefix)
mention.GetTags('🔥', "I'm on 🔥fire!")
Should match fire, but doesn't match anything.
fire
Failing test case:
diff --git i/mention_test.go w/mention_test.go index 423ed9b..904af66 100644 --- i/mention_test.go +++ w/mention_test.go @@ -139,6 +139,9 @@ func (s *MentionSuite) TestGetTags(c *C) { // use default terminators c.Assert(GetTags('@', "hello @test"), DeepEquals, []Tag{{'@', "test", 6}}) + + // Test multibyte start characters. + c.Assert(GetTags('🔥', "I'm on 🔥fire!"), DeepEquals, []Tag{{'🔥', "fire", 6}}) } func BenchmarkGetTags(b *testing.B) {
I'm a bit busy , if you don't mind you can help with implementation. I will check this out when I get a chance.
Since
termIndexes
is a[]int
and some "raw" byte indexes are accessed (e.g.str[t+1] == byte(prefix)
), multibyte prefixes won't work. This:Should match
fire
, but doesn't match anything.Failing test case: