semgrep / ocaml-tree-sitter-core

OCaml code generator and runtime library for handling typed tree-sitter CSTs
GNU Lesser General Public License v2.1
20 stars 2 forks source link

Some tokens have the wrong string content #5

Open aryx opened 3 years ago

aryx commented 3 years ago

See https://returntocorp.slack.com/archives/C01NXGX2EHZ/p1625677111141200

[pad@yrax yy (vue_p2)]$ cat ~/hello.vue 
<script>
hello
world
</script>
[pad@yrax yy (vue_p2)]$ semgrep-core -lang vue -dump_tree_sitter_cst ~/hello.vue 
+ /home/pad/yy/_build/default/src/cli/Main.exe -lang vue -dump_tree_sitter_cst /home/pad/hello.vue
[0.104  Info       Main.Dune__exe__Main ] loaded log_config.json
[0.104  Info       Main.Dune__exe__Main ] Executed as: /home/pad/yy/_build/default/src/cli/Main.exe -lang vue -dump_tree_sitter_cst /home/pad/hello.vue
[0.104  Info       Main.Dune__exe__Main ] Version: semgrep-core version: v0.57.0-26-g6a669dcb-dirty, pfff: 0.42
|   Script_elem
|   |   "<"
|   |   script
|   |   ">"
|   |   helloworld
|   |   "</"
|   |   script
|   |   ">"
mjambon commented 3 years ago

[note to self] I suspect it has to do with Src_file.get_region and the "safe" substring extraction functions in that module of the runtime library.