Syntax highlighting for #_ reader macro

[metasoarous]

This reader macro effectively comments out the following form; it would be nice if there was syntax highlighting to reflect this. I realize this might be a bit challenging, since a form might span multiple lines, but it would still be helpful.

[guns]

Hello @metasoarous 1year+ later!

The #_ reader macro and comment macro used to be treated as comments, but a side effect of this was that the rich structural information from the syntax engine was unavailable. This information can be used by S-expression and REPL plugins to select and evaluate forms, so the behavior was reverted.

Thanks for the suggestion!

[SevereOverfl0w]

When I use the reader macro, It's unusual for me to want to evaluate that.

It's common to see this kind of usage of the reader macro:

(defn foo [a b])

(foo a #_(+ 1 1) b)

Where having the element + macro makes it significantly more readable. If I want to evaluate a region as I am repl-ing along, I tend to use the (comment) macro.

I looked through the commit history, and couldn't find a syntax highlighting which would grey out both the prefix and the element it was attached to. I was hoping you might be able to assist me with a line I could add to my vimrc that would allow me to match for that case.

[guns]

Hi @SevereOverfl0w,

I looked through the commit history, and couldn't find a syntax highlighting which would grey out both the prefix and the element it was attached to. I was hoping you might be able to assist me with a line I could add to my vimrc that would allow me to match for that case.

Looks like I was mistaken. The original VimClojure syntax file simply matched #_ as a comment without handling the form that follows.

Matching #_-commented forms is a tall order for vim's syntax engine. For instance, how do you properly highlight the following without a proper parser?

> (list #_#_ 1 2 3)
(3)

Regardless, I sympathize with your desire to highlight your code to taste.

To handle the common case of a single #_, you will need to match #_ + atoms and define at least four syntax regions to deal with compound forms + strings:

" WARNING: The following code is incorrect
syntax match  clojureComment "\v#_[ \t\r\n]*[^()\[\]{} \t\r\n]+"
syntax region clojureComment start="\v#_[ \t\r\n]*\(" end=")"
syntax region clojureComment start="\v#_[ \t\r\n]*\[" end="]"
syntax region clojureComment start="\v#_[ \t\r\n]*\{" end="}"
syntax region clojureComment start="\v#_[ \t\r\n]*\"" end="\""

I don't think this is production quality, but you might find it satisfactory.

[SevereOverfl0w]

First of all, thank you for taking the time to put that code together for me. That's really great. I really appreciate it. Sorry to stretch you further, but it doesn't seem to match the parens though, so:

#_(defn foo
   []
   (anything))

Unfortunately breaks it, the final parens aren't considered a comment. This might be a better demo, as one of the parts of the body highlights, and the other doesn't.

#_(defn foo
  []
  (a-comment)
  (not-acomment))

It's not clear to me how I should fix this problem though. I'm happy to try figure it out and report back if you can point me in the right direction?

I just did a quick scope of highlighting with emacs, and it seems to do nothing extra in the #_#_ case. I didn't even know that was possible before! Seems dangerous. Not a case I'd be trying to catch, but I understand your reluctance.

[guns]

Unfortunately breaks it, the final parens aren't considered a comment. This might be a better demo, as one of the parts of the body highlights, and the other doesn't.

Yes, I forgot to add recursive matching. :)

Here is a somewhat more robust implementation:

" WARNING: This code has the following KNOWN deficiencies:
"
"   · Consecutive #_ macros are not matched correctly:
"       (list #_#_ 1 2 3) ;; => (3)
"   · Bracket character literals within #_ comment regions break syntax:
"       #_[\a \[ \] \b] ;; broken
"   · Compound forms preceded by reader metacharacters are unmatched:
"       #_'(α β γ) ;; broken
"   · Atomic forms preceded by reader metacharacters + whitespace are unmatched:
"       #_' foo ;; broken
"
syntax match  clojureCommentAtom            /\v#_[ \t\r\n]*[^()\[\]{} \t\r\n]+/
syntax region clojureCommentListContained   start=/(/  end=/)/ contains=clojureCommentListContained,clojureCommentVectorContained,clojureCommentMapContained,clojureCommentStringContained contained
syntax region clojureCommentVectorContained start=/\[/ end=/]/ contains=clojureCommentListContained,clojureCommentVectorContained,clojureCommentMapContained,clojureCommentStringContained contained
syntax region clojureCommentMapContained    start=/{/  end=/}/ contains=clojureCommentListContained,clojureCommentVectorContained,clojureCommentMapContained,clojureCommentStringContained contained
syntax region clojureCommentStringContained start=/"/  skip=/\v\\\\|\\"/ end=/"/ contained
syntax region clojureCommentList            matchgroup=clojureCommentDelimiter start=/\v#_[ \t\r\n]*\(/ end=/)/ contains=clojureCommentListContained,clojureCommentVectorContained,clojureCommentMapContained,clojureCommentStringContained
syntax region clojureCommentVector          matchgroup=clojureCommentDelimiter start=/\v#_[ \t\r\n]*\[/ end=/]/ contains=clojureCommentListContained,clojureCommentVectorContained,clojureCommentMapContained,clojureCommentStringContained
syntax region clojureCommentMap             matchgroup=clojureCommentDelimiter start=/\v#_[ \t\r\n]*\{/ end=/}/ contains=clojureCommentListContained,clojureCommentVectorContained,clojureCommentMapContained,clojureCommentStringContained
syntax region clojureCommentString          matchgroup=clojureCommentDelimiter start=/\v#_[ \t\r\n]*"/  skip=/\v\\\\|\\"/ end=/"/
highlight link clojureCommentDelimiter       clojureComment
highlight link clojureCommentAtom            clojureComment
highlight link clojureCommentListContained   clojureComment
highlight link clojureCommentVectorContained clojureComment
highlight link clojureCommentMapContained    clojureComment
highlight link clojureCommentStringContained clojureComment
highlight link clojureCommentList            clojureComment
highlight link clojureCommentVector          clojureComment
highlight link clojureCommentMap             clojureComment
highlight link clojureCommentString          clojureComment

This should be fairly easy to understand after reading the syn-region help topic (:help syn-region). The contains=, contained, and matchgroup= modifiers are particularly important to apprehend.

If you would like to tackle any of the problems mentioned in the comment header, I would suggest creating the following command:

" http://vimcasts.org/episodes/creating-colorschemes-for-vim/
command! -bar SynStack call <SID>SynStack() "{{{1
function! s:SynStack()
    if exists("*synstack")
        echo map(synstack(line('.'), col('.')), 'synIDattr(v:val, "name")')
    endif
endfunction

This command shows the syntax stack at the cursor position and is invaluable for debugging syntax definitions. Bind this to a normal mapping and have fun!

… the #_#_ case. I didn't even know that was possible before! Seems dangerous. Not a case I'd be trying to catch, but I understand your reluctance.

I believe this trick is more commonly known than a reasonable person might first expect. A quick reading of LispReader.java reveals that the form ignored by #_ is consumed by a recursive call to the same top-level read method used to read any Clojure code. This means, of course, that #_#_ pushes two read calls onto the stack, effectively discarding the next two forms.

Also, I've seen it posed as a koan on IRC channels for understanding the Clojure reader.

[SevereOverfl0w]

Woah, you've gone absolutely above and beyond any expectations here. Thank you so much for your help. This is absolutely brilliant. I really appreciate this.

I'm going to take some time to read through those help files and make sure I understand as much of this as possible. There's no way I would have got this on my own.

[zendevil]

I copied all the commands from https://github.com/guns/vim-clojure-static/issues/60#issuecomment-250875484 to nvim init.vim but don't see the #_ comment "commented out"

[axvr]

@zendevil, I recommend switching to https://github.com/clojure-vim/clojure.vim. It is a fork/continuation of Clojure-vim-static which syntax highlights the comment reader macro (#_) amongst other improvements.

The only downside is that the comment reader macro highlighting in Clojure.vim only highlights the first form as a comment. For example this is highlighted correctly:

#_(defn foo [x]
    (println x))

But this is not (stacked #_#_):

{:foo 1
 #_#_:bar 2}

(If anyone knows how to implement the second case, please open a PR as it'd be greatly appreciated. The related issue can be found here: https://github.com/clojure-vim/clojure.vim/issues/17)