search

microsoft / vscode

Visual Studio Code
https://code.visualstudio.com
MIT License
164.16k stars 29.28k forks source link

Universal Language Injections #1751

Closed felixfbecker closed 3 years ago

felixfbecker commented 8 years ago

Language injections is a feature of IDEs like Webstorm and PHPStorm that auto-detects when a language is use inside another language and then shows syntax highlighting for that injected language. VS Code currently supports this in a few places:

But there are many other contexts where this would be helpful, that are currently not supported and impossible to all hardcode:

All of these examples currently don't get colored and receive no autocompletion. It would be awesome if VS Code was able to automatically detect languages in any context (like string literals, HTML attributes and tags) by some language patterns, like < > tags for HTML/XML, keywords like SELECT, INSERT, etc. for SQL or template string tags like gql for GraphQL. This could be contributed by extensions or language servers.

This would be one of the best features VS Code could offer.

michaelblyons commented 5 years ago

@connorshea Right, but that's the Markdown language defining all of those. As I understand it, this request is for YourNewLanguage not only to color itself, but also inject custom ``` ynl code-fences with highlighting and IntelliSense into Markdown... without submitting PRs to the main Markdown repo.

nitti87 commented 5 years ago

As a disabled person, this would help me alot, I really hope that you (vscodes-guys) are fixing this.

alexbot commented 5 years ago

In the meantime, for html inside js I am ok with extensions like lit-html. It's a compromise I have to live with since it doesn't appear this one will be moving anytime soon.

Here is how it is working for me

jtcownm9d7

Tanja-4732 commented 5 years ago

This would be one of the best new features I could imagine. This would put VSC far ahead of its competition.

While the code highlighting works in Markdown, we still need the language features (for example inside of Markdown code fences).

A feature similar to this could be found in IntelliJ (I think it's even available in the open source community edition).

kevinpickett commented 5 years ago

How does this still not exist. The PHP Storm implementation is fantastic and little things like this keep driving me back.

onetrev commented 5 years ago

For anyone looking for support for other languages in .php files, PHP Intelephense extension has "HTML request forwarding allowing for a full HTML/Javascript/CSS/PHP development experience".

There is an open bug right now with some folks having issues with it (mostly on Windows machines), but with that fixed it will work for everyone and provide what most people here seem to be looking for.

garygreen commented 5 years ago

Just would like to echo my support for this. It would help support Laravel Blade files, single Vue Components, React, etc.

It kinda depresses me to see Microsoft heavily working automatic javascript "code fixing" and refactoring tools (which although great) takes focus away from essential features that should be in any advanced editor and would have a huge impact on every community.

Still hoping more focus will be made but given that the issue is 3 years old now it's pretty much clear nothing's ever going to happen at this point. 😢

Nowaker commented 5 years ago

This is called "Language injection" in JetBrains world (IDEA, Rubymine, Pycharm, and so on) and has existed there since forever. Docs: https://www.jetbrains.com/help/idea/using-language-injections.html

Note: this is mostly automatic, IntelliJ often detects the language in strings.

MindFluid commented 5 years ago

I could really use this feature for CSS/HTML injection inside of .xsl files... Like others are echoing here, JetBrains supports this quite easily, and with great config, out-of-the-box.

I've tried numerous file-association work arounds, but cannot easily implement this. Ultimately, I don't think we should have to rely on community extensions for this -- it should be baked into product.

Language injection is a must!

Ladvien commented 5 years ago

Shesh, I'm writing an extension for Looker's SQL markup language, raw SQL is woven throughout the whole language. This is pretty critical for further development.

ccapndave commented 5 years ago

This is definitely supported (https://code.visualstudio.com/api/language-extensions/syntax-highlight-guide#embedded-languages), but I can't figure out how to actually implement it in practice.

arijoon commented 5 years ago

@ccapndave It looks like it is supported but no docs. I have used the features from grammar setup and it correctly recognizes language in mixed file, and does syntax highlighting (verified by inspecting scope), however no intellisense or formatting possible, I have found this in the docs (under special case) however I cannot find how to actually implement it, maybe it's a feature that isn't implemented yet: https://code.visualstudio.com/api/language-extensions/overview#embedded-languages

KamasamaK commented 5 years ago

@ccapndave @arijoon I've found that the embedded languages feature you're using only affects client-side functionalities such as language snippets and language configuration. The APIs and LSP still rely on the document's language ID.

Soundvessel commented 5 years ago

This ended up being a dealbreaker keeping me from moving away from JetBrains WebStorm. We use SCSS with JSX in our web applications. WebStorm supports most of the SCSS features with a simple comment. Example:

import React from 'react'
import { bool, node, oneOf, string } from 'prop-types'
import classNames from 'classnames'
// eslint-disable-next-line import/no-extraneous-dependencies
import scss from 'styled-jsx/css'
import SiteHeader from './SiteHeader'
import SiteFooter from './SiteFooter'

Layout.propTypes = {
  bodyStyle: oneOf(['default', 'center-xy']),
  children: node.isRequired,
  // turns on footer navbar dark mode and applies fnavbarDarkColor default in <SiteFooter/>
  fnavbarDark: bool,
  // use to replace default utilities, good for changing padding on specific pages
  mainClassNames: string,
  // turns on navbar dark mode and applies navbarDarkColor default in <SiteHeader/>
  navbarDark: bool,
  hideFooter: bool,
  hideHeader: bool,
}

Layout.defaultProps = {
  bodyStyle: 'default',
  fnavbarDark: false,
  mainClassNames: 'py-60', // default <main/> y padding
  navbarDark: false,
  hideHeader: false,
  hideFooter: false,
}

/* language=SCSS */
const globalStyles = scss.global`

  // Setup <main> to fill available browser height keeping footer at bottom

  html, body {
    height: 100%;
  }

  #__next {
    height:         100%;
    display:        flex;
    flex-direction: column;
  }
`

/* language=SCSS */
const styles = scss`

  @import "jsx-helper";

  .page__main {
    flex: 1 0 auto; // footer sticks to bottom of short pages
  }

  // 'center-xy' Layout
  // centers content vertically and horizontally in <main>
  .page__main--center-xy {
    display:         flex;
    justify-content: center;
    align-items:     center;
  }
`

function Layout({
  bodyStyle,
  children,
  hideHeader,
  hideFooter,
  fnavbarDark,
  mainClassNames,
  navbarDark,
}) {
  return (
    <>
      {!hideHeader && <SiteHeader navbarDark={navbarDark} />}
      <main
        role="main"
        className={classNames(
          'page__main',
          `page__main--${bodyStyle}`,
          mainClassNames
        )}
      >
        {children}
      </main>
      {!hideFooter && <SiteFooter fnavbarDark={fnavbarDark} />}
      <style jsx global>
        {globalStyles}
      </style>
      <style jsx>{styles}</style>
    </>
  )
}

export default Layout
rmunn commented 4 years ago

According to https://github.com/microsoft/vscode/wiki/Issues-Triaging issues with more then 20 upvotes go in the Backlog milestone, not Backlog Candidates, yet this issue got assigned (misassigned?) to Backlog Candidates despite having over 300 upvotes. @aeschli Was this done on purpose, or was it a bot bug of some kind?

aeschli commented 4 years ago

The problem with this issue is that is is overly broad, covering many feature areas (syntax highlighting of embedded languages, rich language features) and languages.

There's, from our experience, no universal way of providing support for embedded languages. That's why this issue only serves as a bucket for ideas and scenarios to not lose all the feedback and requests.

ricardosantos79 commented 4 years ago

was looking for a way for highlighter of javascript inside python file, sad this was not implemented, it could work as some keyword combo for example using host language comment system: python: 

# start language=javascript
"""
//highlighed javascript
"""
# end language
nitti87 commented 4 years ago

@aeschli This works in phpstorm

adammichaelwood commented 4 years ago

I think "universal" suggests not that every language and feature is covered, but that the basic infrastructure is in place.

We have an editor that has some way of dealing with a file in language X. Yes? And there's a way to define new languages without building the whole thing all over again. Yes? So why is "make it so two supported languages can exist in the same file" (for whatever support means for a single language) an overly broad feature request?

I understand that there are likely advanced language-specific IDE features that might be tough to deal with. But that isn't really what we're talking about here, is it?

Given that some version of this feature exists in other editors, I'm confused about what makes this impossible here. Do we just need a more clear, minimally viable definition of what constitutes "universal language injection"?

brandonkal commented 4 years ago

Is there anything I can do to get this working? I wrote a "yaml in JS template literal" extension that fixes highlighting. Naturally there is no intellisense there. I imagine this would be a big task.

What I see here is a bunch of "I can't believe this doesn't exist yet" and +1 type comments.

What we really need now is a spec on how this would work and some pointers from the Microsoft team on general direction on where in this codebase to look. Perhaps this requires changes to the LSP?

I imagine something like this where scope is either regex or tmLanguage scope:

{
    "embeddedLanguages":[
        {
            "parent":"typescript",
            "children":{
                "scope":"tagged template literal docker",
                "language":"Dockerfile"
            }
        }
    ]
}

Would that be a good way to handle things? Even there I see some limitations.

The important thing is that users shouldn't have to install extra extensions for the meriad of syntax combinations. It is somewhat easy to contribute an embedded TM language (though even then everything is not 100%). It should be just as easy to embed other languages.

Use cases:

Betanu701 commented 4 years ago

I am not sure if this helps anyone, but I wrote an extension using injection grammars. It utilizes the html language grammar as the parent but then I inject custom tags to allow javascript in between the tags.

It is not perfect, but it allows one to use the existing functionality that is out there without needing to write an entire language server. If there is a "parent" that you can utilize, it is not too difficult to get something similar up and running.

You can find my current iterations here

vscodebot[bot] commented 4 years ago
:slightly_smiling_face: This feature request received a sufficient number of community upvotes and we moved it to our backlog. To learn more about how we handle feature requests, please see our [documentation](https://aka.ms/vscode-issue-lifecycle). Happy Coding!
JoeyRxy commented 4 years ago

yes! SQL in java and php. java in HTML(JSP).

ultraGentle commented 4 years ago

Just to run with the tagged-template-literal idea above, and in the spirit of asechli's Nov. 19 comment that this thread should serve as a "bucket of ideas" —

Use-case for user-tagged simultaneous multi-language support 

// in .js file
myElement.innerHTML = /* start html, js */ `
<div class="Html-Language-Features-Needed-Here"> 
${ andJavascriptFeaturesHere } </div>
`/* end html, js */

and another example of when this might occur as language-in-language-in-language
(HTML string --> in JS template literal --> in HTML doc)

<!-- in .html file -->
<script>
myElement.innerHTML = `
<div class="Html-Features-Would-Be-Useful-Here"> 
${andJavascriptFeaturesHere} </div>
` 
</script>

So, as a user, I would't mind having to manually tell the editor when I want which language, if it would mean that I can get intellisense for any/all languages where useful.

This might make the feature easier to implement, too, since it wouldn't need to be as "smart," in detecting all possible language-in-language-(in-language) cases.

egriff38 commented 4 years ago

@ultraGentle this would be fantastic, but it seems like having to accommodate a variety of language features for intellisense might make this tricky to inline. For example, I am currently working on a project where it would be useful to inline webworker code:

// exterior js
var blob = new Blob([/* start js, js */`
  self.postMessage("hello world")
  // web worker js
`/* end js, js */], { type: "text/javascript" })

var worker = new Worker(window.URL.createObjectURL(blob));
...

So if I wanted vscode to give me all the wonderful things intellisense offers, I must tell it

  1. The inner and outer JS contexts are isolated, and
  2. The inner context has a Worker instance for self and the outer has access to the DOM.

This may be contrived as trivial, except that my project uses typescript instead of javascript, which can be very picky about these sorts of things.

Great idea though!

egriff38 commented 4 years ago

On the subject of typescript though, it would be neat to leverage type aliasing somehow to render strings properly:

import {HTMLString} from "@fictitious/string_interpolate"

const link: HTMLString = `<a href="#">link</a>` // would syntax highlight
// or
const par = `<p class="big"> foo bar baz</p>` as HTMLString // would also highlight
brandonkal commented 4 years ago

Please don't do that. Write your worker in its own module. Rather than inventing a comment syntax, use a pass-through tagged template function. i.e. html`...`

ultraGentle commented 4 years ago

@brandonkal to clarify, the comment syntax above is just to provide a clear example of the desired behavior. Implementation would need to be more carefully considered.

@xepher0coded thanks! Indeed, not entirely practical as described; just wanted to voice the idea in case someone had an ingenious way of implementing it. Possibly as an editor-level feature rather than something in-code, like logpoints vs console.log statements.

Anyway, as originally mentioned, just adding to the bucket o' ideas, here.

ghost commented 4 years ago

Yes and "css.remoteStyleSheets" suggestions for html in xml file.

vadimmos commented 4 years ago

Syntax highlighting and intellisence in JavaScript template strings containing html would be one of the best things that vs-code could offer. WebStorm has been able to do this for a long time, and this is why many developers do not change it to vs-code. This feature is now required by almost all web developers. Almost all frameworks require working with template strings containing html, and sometimes this is required for vanilla JS.

nitti87 commented 4 years ago

@vadimmos Exactly what I want

kanlukasz commented 4 years ago

I look forward to it. At the moment, I can't even use Intellisense for HTML classes in php files where just pure HTML is It's ridiculous.

onetrev commented 4 years ago

While Universal Language Injections would be amazing, for PHP at least @kanlukasz you can get Intellisense!

This extension (PHP Intelephense), since it's more recent updates, works perfectly for HTML in PHP! Definitely works better than other PHP Intellisense options I've tried. I'm not entirely sure how the creator (Ben Mewburn) got around the limitations but I'm glad he did.

jens1o commented 4 years ago

While Universal Language Injections would be amazing, for PHP at least @kanlukasz you can get Intellisense!

This extension (PHP Intelephense), since it's more recent updates, works perfectly for HTML in PHP! Definitely works better than the original PHP Intellisense option. I'm not entirely sure how the creator (Ben Mewburn) got around the limitations but I'm glad he did.

And it is not FOSS. So not even close to an option for me. Thank you for your advertisment though, not solving the general problem at all.

onetrev commented 4 years ago

Sad to see the negativity @jens1o. There are premium features, it's kind of nice for people to actually be able to make a living with their time, but the main functionality is fully free, I'm looking at the Git repo right now. https://github.com/bmewburn/vscode-intelephense

erichiller commented 4 years ago

Also, for Markdown this would be very much appreciated. (as in other languages inside markdown files using triple backtick)

Logerfo commented 4 years ago

AFAIK this already works for markdown because it's implemented in the markdown language engine for vscode

michaelblyons commented 4 years ago

@Logerfo I know this thread is very long, but I have written about the differences before which you can find if you look for them.

Also, to everyone else: Please stop adding "+1"-like comments. Upvote the issue and relevant comments you agree with, but only post a comment yourself if there's something new to add. That will shorten the cognitive load for people to scan the whole thread.

vadimmos commented 4 years ago

Inside html, between tags, js is supported at a very low level. JsDoc doesn't work here, and there are no hints about js in linked scripts. Inside js, there is no support for anything else at all. Support for one code within another code is a very necessary technology, especially in the web.

leeoniya commented 4 years ago

well, the good news is that CodeMirror 6 just entered beta:

https://marijnhaverbeke.nl/blog/codemirror-6-beta.html

trelljaret commented 4 years ago

I use python inside bash a lot. Take this as an example:

#!/usr/bin/env bash

python3 <<EOL
import re

with open("inside.sh", "r") as fp:
    data = fp.read()

data = re.sub(r"enablet=\".*?\" ;", 'enablet="--enable" ;', data, re.IGNORECASE | re.MULTILINE)

with open("inside.sh", "w") as fp:
    fp.write(data)
EOL

git add . ;
git commit -am 'change' ;
git push localenv master ;

Can vscode detect python3 <<EOL until EOL is a python code and syntax highlight and add autocomplete functionality to that? Is there any plugin for this right now?

Sayyiditow commented 4 years ago

I wish they could highlight SQL syntax in XML files, this is a common practice for us.

KamasamaK commented 3 years ago

For anyone interested in checking the language at the active cursor, regardless of whether it's embedded, in the when clause, I created #107989. It's not as nice as API support, but it's something.

matu3ba commented 3 years ago

Please consider porting treesitter for this. Then the only hard part that remains is gluing the parsers together in a meaningful way. Also the languages that are glued together must be annotated somewhere in a standard-conforming way.

This has already been implemented for lua-heredoc, but only for full lines and not inline strings or stuff like that (so the context was pretty simple).

matu3ba commented 3 years ago

I use python inside bash a lot. Take this as an example:

#!/usr/bin/env bash

python3 <<EOL
import re

with open("inside.sh", "r") as fp:
    data = fp.read()

data = re.sub(r"enablet=\".*?\" ;", 'enablet="--enable" ;', data, re.IGNORECASE | re.MULTILINE)

with open("inside.sh", "w") as fp:
    fp.write(data)
EOL

git add . ;
git commit -am 'change' ;
git push localenv master ;

Can vscode detect python3 <<EOL until EOL is a python code and syntax highlight and add autocomplete functionality to that? Is there any plugin for this right now?

you can do this with neovim and treesitter.

Thom1729 commented 3 years ago

FYI, I wrote a Sublime package that does this for JavaScript. The user specifies the behavior they want in the package preferences, e.g.:

{
    "defaults": {
        "My Config": {
            "custom_templates": {
                "tags": {
                    "html": "scope:text.html.basic"
                },
                "comments": {
                    "style": "scope:source.css"
                },
                "lookaheads": {
                    "select\b": "scope:source.sql"
                }
            }
        }
    }
}

The package then produces a dynamically generated syntax definition based on the standard JavaScript def, but with the selected features added. In this example, you'd get language-specific highlighting inside each of the following template literals:

html`<strong>Hello, World!</strong>`;
/* style */`color: red`;
`select * from users`;

I imagine that someone could write a similar extension for VSCode.

vladdeSV commented 3 years ago

This is something I would very much like to have.

Similar to one other comment, when dealing with nested languages (eg. SQL in JS) I usually write the code in its own file with highlighting, then copy the code from there.

xeor commented 3 years ago

vetur have done some work around this. There are some magic in https://github.com/vuejs/vetur/blob/master/server/src/embeddedSupport/embeddedSupport.ts and https://github.com/vuejs/vetur/blob/c97734bc89431d9bbc75f129d9593193d80c1800/server/src/services/projectService.ts#L143 is an example-usage for when it does auto-completion.

I think the vetur-way of doing this is a hack tho, so it might be better waiting for this issue.

This should support both

```css ... ```

style, and

\...\,

aeschli commented 3 years ago

Thanks for all the input and ideas posted. I'm closing this issue as the issue is difficult to act on. The issue started overly broad (and ambitious) and has become even broader over time. While we love hearing ideas and use cases, keeping it open raised expectations that we don't know how to fulfil.

I don't see a 'universal' solution to embedded languages that VS Code could implement. I believe that given the language specific syntaxes and mechanisms that are used, every embedding feature has to be solved in the context of the main language of the file. Generic approaches can solve 80% of the problem but cause frustrations and unsolvable issues for the rest. So want to encourage people to file feature requests against the language extension.

VS Code offers various APIs for extension developers to add support for embedded languages. There are mechanisms to

KamasamaK commented 3 years ago

Fixed the link to the JSON language service.