Add queries

[ghost]

We should add queries, similar to https://github.com/tree-sitter/tree-sitter-python/tree/master/queries which can be used to highlight, fold, indent Fortran source files.

Relevant documentation: https://tree-sitter.github.io/tree-sitter/using-parsers#pattern-matching-with-queries

Query files to add:

• [x] highlights.scm
• [x] indents.scm
• [x] folds.scm
• [ ] locals.scm

[stadelmanma]

@oponkork agreed, we might need to revise some of the code that uses anonymous nodes if I didn't explicitly call out the node. I think I did for most "operators" (e.g. ==, !=, etc.) but there may have been areas that I missed.

Anonymous Nodes

The parenthesized syntax for writing nodes only applies to named nodes. To match specific anonymous nodes,
you write their name between double quotes. For example, this pattern would match any binary_expression
where the operator is != and the right side is null:

(binary_expression
  operator: "!="
  right: (null))

We will also want to make sure highlighting gets tested, see: https://tree-sitter.github.io/tree-sitter/syntax-highlighting#unit-testing. I didn't see an explicit way to test the "queries" not used for syntax highlighting.

[ghost]

@oponkork agreed, we might need to revise some of the code that uses anonymous nodes if I didn't explicitly call out the node. I think I did for most "operators" (e.g. ==, !=, etc.) but there may have been areas that I missed.

@stadelmanma I have made some changes to the grammar to handle keywords. See #51

Anonymous Nodes

The parenthesized syntax for writing nodes only applies to named nodes. To match specific anonymous nodes,
you write their name between double quotes. For example, this pattern would match any binary_expression
where the operator is != and the right side is null:

(binary_expression
  operator: "!="
  right: (null))

We will also want to make sure highlighting gets tested, see: https://tree-sitter.github.io/tree-sitter/syntax-highlighting#unit-testing. I didn't see an explicit way to test the "queries" not used for syntax highlighting.

I have been testing highlighting and folds queries using neovim-master and nvim-treesitter.

[mrossinek]

@oponkork I would love to help develop and/or test those queries! Would you be able to share what you have currently in a branch? I am also using neovim master and with nvim-treesitter.

Btw: do you happen to know how nvim-treesitter handles queries? If I understand the structure correctly it provides its own query files but it is possible to provide custom queries on the runtimepath. Wouldn't it make sense for nvim-treesitter to use the queries provides by a tree-sitter plugin (such as this one) if they are provided?

E.g. right now, the Python queries differ between nvim-treesitter and tree-sitter-python which could/should be avoided, don't you think?

[stadelmanma]

@mrossinek that sounds like it would be ideal, the only catch would be if the highlighting node naming conventions differ.

[ghost]

@oponkork I would love to help develop and/or test those queries! Would you be able to share what you have currently in a branch? I am also using neovim master and with nvim-treesitter.

I have pushed whatever little work I had done to oponkork:queries. Hope it helps.

Btw: do you happen to know how nvim-treesitter handles queries? If I understand the structure correctly it provides its own query files but it is possible to provide custom queries on the runtimepath. Wouldn't it make sense for nvim-treesitter to use the queries provides by a tree-sitter plugin (such as this one) if they are provided?

I am not very well informed on this part. IIRC the nodes are the same because they come from the parser, but the capture names will depend on the implementation. There was a discussion here, check it out: https://nvim-treesitter.zulipchat.com/#narrow/stream/252271-general/topic/grammar.20and.20highlights.20for.20sparql

You might need to sign in though.

[mrossinek]

Thanks for the links! I didn't think that the queries would be different depending on the editor/interpreter/or whomever wants to use them. I thought there would be strict guidelines to allow a single effort of writing these to be used across the board. But then again, I didn't have a close look at them at all, yet.

@oponkork thanks for pushing the WIP queries! In view of the above: how do you use those? Do you use them in conjunction with nvim-treesitter? Or can they be used as a standalone feature (?) with this project? I think the above confused me a little because now I don't really understand what the parser needs queries for if (in the end) the editor will implement queries for itself?

[ghost]

Thanks for the links! I didn't think that the queries would be different depending on the editor/interpreter/or whomever wants to use them. I thought there would be strict guidelines to allow a single effort of writing these to be used across the board. But then again, I didn't have a close look at them at all, yet.

Yep, I also expected the same, but haven't invested much time in it yet. I have been using the queries with nvim-treesitter as follows:

Copy the scm files to the nvim-treesitter/queries/fortran/ directory. For me the full path is /home/oponkork/.local/share/nvim/plugged/nvim-treesitter/queries/fortran. Add the following to nvim-treesitter/lua/nvim-treesitter/parsers.lua

list.fortran = {
  install_info = {
    url = "https://github.com/stadelmanma/tree-sitter-fortran",
    files = { "src/parser.c", "src/scanner.cc" }
  },
  used_by = { "fortran" },
  maintainers = {"@stadelmanma"},
}

Run :TSInstall fortran and open a fortran file. It should start highlighting (which is incomplete at the moment). For folding you have to add the following somewhere in your vimrc (after you load nvim-treesitter).

    set foldmethod=expr
    set foldexpr=nvim_treesitter#foldexpr()

Let me know if it works for you.

You can also test highlighting as mentioned in the link below, but I did not try it out.

https://tree-sitter.github.io/tree-sitter/syntax-highlighting

[mrossinek]

Thanks for the detailed instructions! I had already figured out how to get the parser installed and I have now integrated the queries into my dotfiles (https://github.com/mrossinek/dotfiles/commit/24eaa89df5154f8d67653bd222fd6c10f6cb2578). I will keep you updated if I make some progress but lately I am too busy with other things so it may be a while before I can get back to you on this.

[stadelmanma]

@oponkork is this issue resolved with #58?

[ghost]

Yes, mostly. Queries need little more work (mostly adding few keywords and rules which I might have missed), but the major portion of the work is done.