raw string literals incorrectly detected and terminated

[krux02]

Well aparently github doesn't do it correctly either, so I guess it isn't trivial. But this file should highlight correctly.

import macros

macro myRaw(arg: string): string =
  result = newLit(arg.strVal)

echo      "  foo\\\"bar \\"  # a comment "
echo myRaw"  foo\""bar \"    # a comment "
echo     r"  foo\""bar \"    # a comment "
echo    """  foo\"bar \"""   # a comment "

echo ""       # a comment
echo r""      # a comment
echo myRaw""  # a comment

• raw strings should end at \", where normal strings shouldn't.
• <ident>"str ..." should be parsed as raw string literal.

[yuutayamada]

Is there document for <ident>"str ..."? I'm not sure Nim's spec for that case.

[yuutayamada]

I just discovered python's raw string literal question (https://stackoverflow.com/questions/647769/why-cant-pythons-raw-string-literals-end-with-a-single-backslash) Do you still think the end of \" would be possible on Nim?

[krux02]

Well, that is a discussion about the internal structures of python, that is not lexical analysis in Nim works. In Nim a raw string literal can end in a backslash.

The only argument you can bring is, that it isn't very useful to have a raw string literal end it a backslash and not worth the effort. It is by far easier to forbid it in the language than to add support for it in every editor.

For the <ident>"str ..." case, it is documented here: https://nim-lang.org/docs/macros.html#callsslashexpressions-call-with-raw-string-literal

[yuutayamada]

The only argument you can bring is, that it isn't very useful to have a raw string literal end it a backslash and not worth the effort. It is by far easier to forbid it in the language than to add support for it in every editor.

I don't mind implementing this feature or fix. Give me some time.

I'm not sure regarding the <ident>"str ..." yet. Is the thing always evaluated as raw string if there is no space between identifier and double quotes? or is it depending on macro's implementation?

[krux02]

For the macro argument it is always a raw string literal. If the macro wants to support excaped characters such as \t as tabulators, the macro has to actually substitute the literal "\t" with a tabulator, but that is nothing nim-mode needs to be aware of.

An example of how the raw string literals can be used is here: https://github.com/nim-lang/Nim/blob/devel/compiler/incremental.nim#L111

Just for fun did I implement some fontification for this. I wanted to highlight sql keywords in sql string literals. This is how I did it:

(defun nim--inside-sql-raw-string-literal-p ()
  "checkous if inside a string literal, and if the identifier before the string literal is 'sql'"
  (let* ((ppss (syntax-ppss))
         (pos (nth 2 ppss)))
    (when (and pos (nth 3 ppss))
      (save-excursion
        (goto-char pos)
        (looking-at-p "\\<sql\\>")))))

(defun nim-sql-keyword-matcher (&optional limit)
  "Highlight sql keywords."
  (and
   ;; Skip through files without sql strings fast.
   ;; Simple regex with low false positives.
   (or (nim--inside-sql-raw-string-literal-p)
       (re-search-forward "\\<sql\"+" limit t))
   (let ((case-fold-search t) ; sql is case insesitive
         res)
     (while
         (and
          (setq res (re-search-forward
                     nim--sql-keywords-regexp limit t))
          (not (nim--inside-sql-raw-string-literal-p))))
     res)))

With this pattern support for many more embedded languages can be added. But I am not sure how many of those domain languages emacs can handle at the same time, since they are all activated globally. Maybe there is a better way to query and activate them.

I currently have the idea that a domain language can be registered somehow. Then whenever there is a CallStrLit emacs can ask nimsuggest if there are keywords for this callstrlit and then feed this information to the fontification.

[yuutayamada]

thank you for the clarification. the issue should be fixed now