add support for embedded languages

[gudvinr]

There are few cases in Go where you may want to other languages embedded e.g. in strings. Couple of simple ones that come to mind are:

• Markup languages such as HTML can be used with html/template
• Raw SQL queries for database/sql and other packages that support SQL queries like pgx

There are another cases like embedding Lua scripts. It is useful for writing scripts for Redis. Also handy if you embed Lua VM in your application. Same for other scripting environments.

Documentation for Visual Studio Code has some information about this topic.

Implementation choice might require support from LSP side but it depends so I am filling issue here and not in golang/go.

[hyangah]

@pjweinb @stamblerre

I expected a similar approach for https://github.com/golang/vscode-go/issues/609 except that for Go templates support, the gopls will handle the texmplate syntax but for other languages, we route that to other language servers or extensions (I am not sure yet about how to detect other languages from the language client and forward to the right service, beyond the trivial cases described in the doc)

Note the current WIP prototype for golang/vscode-go#609 & golang/go#36911 is not actually handling go templates "embedded" in .go files, but focuses more on supporting go templates in a separate file.

With go1.16's embedding support, I hope keeping other languages in separate files would be easier.

Ideas and contributions are welcome.

[gudvinr]

I think maybe naive Bayes classifiers might do the trick.

Uses of embedded languages in string literals are limited for compiled and statically typed languages like Go.
In my opinion, most of the time it is safe to make following assumptions:

• string literals are short so they easily readable by other people when outside of generated code
• list of languages used in such strings should be quite small
• language inside string literal can be identified using keywords specific to language

So, with these limitations it should be possible to determine which languages people frequently embed in strings and compute classifier parameters for these languages.

[a-h]

JavaScript supports the concept of template literals, which are defined by adding a "tag" before the backtick that opens the string literal. This makes it easier for tools to determine what the content in the string is and apply appropriate formatting / linting etc.

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_literals

These tags can also be functions that carry out work on the template. I've used the https://github.com/apollographql/graphql-tag before.

A change to the Go language to enable a similar syntax for multi-line string literals might be useful here.

Goland supports type hints in template files by the use of a comment https://blog.jetbrains.com/go/2018/12/14/go-templates-made-easy/

[SealOfTime]

I believe we could look at the variable, parameter (and the function name) to try and figure out which particular language has been embedded. By default stick to the existing in standard library patterns:

1. Function is a Query method of sql.Tx or sql.DB, and the literal is used for parameter "query", then we expect SQL to sit right there
2. String is passed as argument to text/template.Template.Parse, then it's a template
3. html.Template.Parse - it's a HTML with template

Ideally we'd also have a way to define custom matchers, based on regular expressions applied to either variable name, function/method name including the package, perhaps a struct field

This could work through request-forwarding as shown in VSCode example