search

cedricrupb / code_tokenize

Fast tokenization and structural analysis of any programming language
MIT License
44 stars 8 forks source link

The tree-sitter-go can not parse the type of string. #2

Closed GoneZ5 closed 2 years ago

GoneZ5 commented 2 years ago

Hi, when I parse the go function, the string value can not be parsed. Can you help me?

image

image

Thank you in advance!

cedricrupb commented 2 years ago

Hey, sorry for the delayed reply.

Sadly, this is actually a problem of unconventional handling of strings by tree-sitter for Go. Since the strings are represented as interpreted_string_literal instead of string which breaks the string handling during tokenization.

In the current version, there is no easy workaround. However, I am working hard on version 0.2 which will make it much easier to integrate custom behavior for tokenization.

cedricrupb commented 2 years ago

Should be closed in version 0.2.0. Please reopen if necessary.