gernest / mention

Twitter like mentions and #hashtags parser for Go(Golang)
MIT License
107 stars 14 forks source link

Multibyte start characters don't work #16

Open arp242 opened 6 years ago

arp242 commented 6 years ago

Since termIndexes is a []int and some "raw" byte indexes are accessed (e.g. str[t+1] == byte(prefix)), multibyte prefixes won't work. This:

mention.GetTags('🔥', "I'm on 🔥fire!")

Should match fire, but doesn't match anything.

Failing test case:

diff --git i/mention_test.go w/mention_test.go
index 423ed9b..904af66 100644
--- i/mention_test.go
+++ w/mention_test.go
@@ -139,6 +139,9 @@ func (s *MentionSuite) TestGetTags(c *C) {

        // use default terminators
        c.Assert(GetTags('@', "hello @test"), DeepEquals, []Tag{{'@', "test", 6}})
+
+       // Test multibyte start characters.
+       c.Assert(GetTags('🔥', "I'm on 🔥fire!"), DeepEquals, []Tag{{'🔥', "fire", 6}})
 }

 func BenchmarkGetTags(b *testing.B) {
gernest commented 6 years ago

I'm a bit busy , if you don't mind you can help with implementation. I will check this out when I get a chance.