search

tree-sitter / py-tree-sitter

Python bindings to the Tree-sitter parsing library
https://tree-sitter.github.io/py-tree-sitter/
MIT License
891 stars 106 forks source link

[bug] Cannot build a library in a directory containing two parsers. #189

Closed Freed-Wu closed 8 months ago

Freed-Wu commented 8 months ago

https://github.com/MichaHoffmann/tree-sitter-hcl has two parsers: hcl and terraform.

>>> Language.build_library("/dev/shm/a.so", ["vendor/tree-sitter-hcl", "vendor/tree-sitter-hcl/dialects/terraform"])
/nix/store/idiaraknw071d20nlqp49s18gbvw4wa0-binutils-2.40/bin/ld: /tmp/tmpl0bxz4f_tree_sitter_language/vendor/tree-sitter-hcl/dialects/terraform/src/scanner.o: in function `string_new':
scanner.c:(.text+0x0): multiple definition of `string_new'; /tmp/tmpl0bxz4f_tree_sitter_language/vendor/tree-sitter-hcl/src/scanner.o:scanner.c:(.text+0x0): first defined here
collect2: error: ld returned 1 exit status
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/wzy/.local/lib/python3.11/site-packages/tree_sitter/__init__.py", line 118, in build_library
    compiler.link_shared_object(
  File "/nix/store/3rv445i5nn55xlkvmk2ai7spv0vrn0c9-python3-3.11.6-env/lib/python3.11/site-packages/setuptools/_distutils/ccompiler.py", line 752, in link_shared_object
    self.link(
  File "/nix/store/3rv445i5nn55xlkvmk2ai7spv0vrn0c9-python3-3.11.6-env/lib/python3.11/site-packages/setuptools/_distutils/unixccompiler.py", line 277, in link
    raise LinkError(msg)
distutils.errors.LinkError: command '/run/current-system/sw/bin/cc' failed with exit code 1
Freed-Wu commented 8 months ago

Similar bug for https://github.com/postsolar/tree-sitter-purescript and https://github.com/tree-sitter/tree-sitter-haskell.

>>> Language.build_library("/dev/shm/a.so", ["vendor/tree-sitter-purescript", "vendor/tree-sitter-haskell"])
/nix/store/idiaraknw071d20nlqp49s18gbvw4wa0-binutils-2.40/bin/ld: /tmp/tmprbdxway9tree_sitter_language/vendor/tree-sitter-haskell/src/scanner.o: in function `state_new':
scanner.c:(.text+0x1906): multiple definition of `state_new'; /tmp/tmprbdxway9tree_sitter_language/vendor/tree-sitter-purescript/src/scanner.o:scanner.c:(.text+0x1906): first defined here
collect2: error: ld returned 1 exit status
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/wzy/.local/lib/python3.11/site-packages/tree_sitter/__init__.py", line 118, in build_library
    compiler.link_shared_object(
  File "/nix/store/3rv445i5nn55xlkvmk2ai7spv0vrn0c9-python3-3.11.6-env/lib/python3.11/site-packages/setuptools/_distutils/ccompiler.py", line 752, in link_shared_object
    self.link(
  File "/nix/store/3rv445i5nn55xlkvmk2ai7spv0vrn0c9-python3-3.11.6-env/lib/python3.11/site-packages/setuptools/_distutils/unixccompiler.py", line 277, in link
    raise LinkError(msg)
distutils.errors.LinkError: command '/run/current-system/sw/bin/cc' failed with exit code 1
milahu commented 8 months ago

these are bugs in the parsers

see also https://github.com/tree-sitter/tree-sitter-html/issues/63 https://github.com/tree-sitter/tree-sitter-html/pull/64

Freed-Wu commented 8 months ago

Yes, https://github.com/grantjenks/py-tree-sitter-languages/pull/53/commits/e38ce310ce9b1328e76e9f06b02f709a69d810a1 disable some languages.

disabled_langs = [
    "vue",  # html, angular
    "angular",  # html
    "purescript",  # haskell, unison
    "unison",  # haskell
    "svelte",  # org
    "terraform",  # hcl
]

When two parsers define same definitions, the symbols should have a namespace-like prefix.

milahu commented 8 months ago

@Freed-Wu

re: https://github.com/NixOS/nixpkgs/issues/263630

https://github.com/milahu/nur-packages/commit/48f3c130ce71bbed0be017800103edcd0c18afd5

this works with all grammars except svelte and vue

milahu commented 8 months ago

https://github.com/milahu/nur-packages/commit/48f3c130ce71bbed0be017800103edcd0c18afd5

fixed in https://github.com/milahu/nur-packages/commit/85dccc20ffb4b88978058d039dcf6b87bb118bbe