webpack/browserify compatible AMD headers

NikGovorov commented 8 years ago

Hello,

Is there any way to change AMD headers to be compatible with webpack/browserify?

Are there any plans to support CommonJS, UMD or ECMAscript 6 for the package and external modules in monaco.d.ts?

MazheM commented 8 years ago

+1 I want to use it in my app, but i cant because use webpack for modules.

alexdima commented 8 years ago

https://webpack.github.io/docs/amd.html

I thought webpack understands AMD code? What more should be done on our side to get the editor loadable with webpack?

NikGovorov commented 8 years ago

I think webpack parses headers(imports/exports) and obviously using the constructs like https://github.com/Microsoft/vscode/blob/master/build/lib/bundle.ts#L278 breaks parsing.

alexdima commented 8 years ago

Well, the editor.main.js that ships is already packed (contains 500+ AMD modules). Is the intent here to repack it (run the editor.main.js + all the javascript files we ship) through webpack? I have not used webpack so I'm not sure what we can do on our side... Should we also ship an unbundled/unpacked version where each AMD module sits in its own file?

MazheM commented 8 years ago

@alexandrudima i tried to use monaco in my script like that:

require(['vs/editor/editor.main'], function() {
    var editor = monaco.editor.create(document.getElementById('container'), {
        value: [
            'function x() {',
            '\tconsole.log("Hello world!");',
            '}'
        ].join('\n'),
        language: 'javascript'
    });
});

But it dosent work because Webpack cant find "edcore.main" for example, because he try to find it like file in "vs/editor" folder. Log:

Module not found: Error: Cannot resolve module 'vs/editor/edcore.main' in src/js/vendor/vs/editor
resolve module vs/editor/edcore.main in src/js/vendor/vs/editor
  looking for modules in bower_components
    bower_components/vs doesn't exist (module as directory)
  looking for modules in src/js/vendor
    resolve 'file' or 'directory' editor/edcore.main in src/js/vendor/vs
      resolve file
        src/js/vendor/vs/editor/edcore.main doesn't exist
        src/js/vendor/vs/editor/edcore.main.web.js doesn't exist
        src/js/vendor/vs/editor/edcore.main.webpack.js doesn't exist
        src/js/vendor/vs/editor/edcore.main.js doesn't exist
        src/js/vendor/vs/editor/edcore.main.json doesn't exist
      resolve directory
        src/js/vendor/vs/editor/edcore.main doesn't exist (directory default file)
        src/js/vendor/vs/editor/edcore.main/package.json doesn't exist (directory description file)
[bower_components/vs]
[src/js/vendor/vs/editor/edcore.main]
[src/js/vendor/vs/editor/edcore.main.web.js]
[src/js/vendor/vs/editor/edcore.main.webpack.js]
[src/js/vendor/vs/editor/edcore.main.js]
[src/js/vendor/vs/editor/edcore.main.json]

Maybe unpacked version will be work.

NikGovorov commented 8 years ago

@alexandrudima, yes I think shipping not bundled amd modules would definitely work for webpack/browserify.

@MazheM before you can see the error I faced you need to remap paths: 'vs': path.join(__dirname, 'node_modules/monaco-editor-core/dev/vs/' https://webpack.github.io/docs/resolving.html

NikGovorov commented 8 years ago

To be totally honest I believe that main npm package should only contain CommonJS version + d.ts with external modules. Bundles for different formats amd, umd etc(and ambient d.ts) should be distributed as bower or jspm packages, or at least as secondary npm packages.

MazheM commented 8 years ago

@NikGovorov Yes, i know it. I placed monaco to src/js/vendor/vs and in webpack config i have modulesDirectories: ['node_modules', 'src/js/vendor'],

Matthias247 commented 8 years ago

I would also appreciate some examples or help on how to get it to run with webpack. I tested the following approaches:

Added to webpack resolve configuration:

resolve: {
    alias: {
        vs: path.resolve(__dirname, 'node_modules/monaco-editor/dev/vs'),
    },

and did

require('vs/editor/editor.main')

in my main module. Alternatively to the second step I also tried putting the following in the plugins section of webpack:

new webpack.ProvidePlugin({
       "monaco": "monaco-editor/dev/vs/editor/editor.main"
    }),

Both end up with missing references to edcore.main and fs (which is referenced by typescriptServices.js - but fs is a node module and shouldn't be available in the browser?)

NikGovorov commented 8 years ago

@alexandrudima is any update on this?

Matthias247 commented 8 years ago

I got monaco working now with webpack and angular2.

What I did: Configure webpack to simply copy the bundled monaco version to my output directory with CopyWebpackPlugin:

plugins: [
    new CopyWebpackPlugin([
        {
            from: 'node_modules/monaco-editor/min/vs',
            to: 'vs',
        }
    ]),
],

and build a monaco angular2 component that loads the AMD loader if required and then loads monaco as a global variable:

import { Component, ViewChild, ElementRef, ViewQuery }
         from '@angular/core';
import { COMMON_DIRECTIVES  } 
         from '@angular/common';

import * as _ from 'lodash';

declare const monaco: any;
declare const require: any;

@Component({
  selector: 'monaco-editor',
  directives: [COMMON_DIRECTIVES],
  template: `
    <div id='editor' #editor class="monaco-editor" style="width: 1000px; height: 1000px">
  `,
})
export class MonacoEditor {
  @ViewChild('editor') editorContent: ElementRef;

  constructor(
  ) {
  }

  ngAfterViewInit() {
    var onGotAmdLoader = () => {
      // Load monaco
      (<any>window).require(['vs/editor/editor.main'], () => {
        this.initMonaco();
      });
    };

    // Load AMD loader if necessary
    if (!(<any>window).require) {
      var loaderScript = document.createElement('script');
      loaderScript.type = 'text/javascript';
      loaderScript.src = 'vs/loader.js';
      loaderScript.addEventListener('load', onGotAmdLoader);
      document.body.appendChild(loaderScript);
    } else {
      onGotAmdLoader();
    }
  }

  // Will be called once monaco library is available
  initMonaco() {
    var myDiv: HTMLDivElement = this.editorContent.nativeElement;
    var editor = monaco.editor.create(myDiv, {
      value: [
        'function x() {',
        '\tconsole.log("Hello world!");',
        '}'
      ].join('\n'),
      language: 'javascript'
    });
  }
}

This should work in a similar fashion for a react component.

It might be problematic if another application part loads an incompatible AMD loader and a commonjs or unbundled version might be easier to integrate, but it works for me now. The plus side with that approach is that all monaco sources and licenses are at a clear path in the output folder.

ebertmi commented 8 years ago

Still, it would be more usable in many projects if we could use it with webpack without the vs/loader.js. I tried to require the monaco-editor-core/vs/editor/editor.main which works until it tries to execute the define function with following error define cannot be used indirect.

egoist commented 8 years ago

just did as what @Matthias247 suggested, but got Module not found: edcore.main, full error log:

ERROR in ./app/~/monaco-editor/dev/vs/editor/editor.main.js
Module not found: Error: Cannot resolve 'file' or 'directory' /Users/egoist/dev/jslab/app/node_modules/monaco-editor/dev/vs/editor/edcore.main in /Users/egoist/dev/jslab/app/node_modules/monaco-editor/dev/vs/editor
@ ./app/~/monaco-editor/dev/vs/editor/editor.main.js 83129:173-266 83136:173-267 83143:0-3357 83144:0-244

Version 0.5.3

fxck commented 8 years ago

Any update? It's basically unusable in angular2 projects as it is.

tugberkugurlu commented 8 years ago

This is a major missing part :disappointed: would be really nice to get it working with webpack without hacking with CopyWebpackPlugin.

rrfenton commented 8 years ago

Tried to implement monaco using systemJS with jspm, which supports AMD modules.

Getting the following error on import:

(index):14 Error: (SystemJS) Cannot read property 'performance' of undefined
    TypeError: Cannot read property 'performance' of undefined
        at AMDLoader (http://localhost:14992/jspm_packages/npm/monaco-editor@0.5.3/dev/vs/loader.js:1552:36)
        at eval (http://localhost:14992/jspm_packages/npm/monaco-editor@0.5.3/dev/vs/loader.js:1650:5)
        at Object.eval (http://localhost:14992/jspm_packages/npm/monaco-editor@0.5.3/dev/vs/loader.js:2228:3)
        at eval (http://localhost:14992/jspm_packages/npm/monaco-editor@0.5.3/dev/vs/loader.js:2230:4)
        at eval (http://localhost:14992/jspm_packages/npm/monaco-editor@0.5.3/dev/vs/loader.js:2231:3)
    Evaluating http://localhost:14992/jspm_packages/npm/monaco-editor@0.5.3/dev/vs/loader.js
    Evaluating http://localhost:14992/jspm_packages/npm/monaco-editor@0.5.3.js
    Evaluating http://localhost:14992/app/app.component.js
    Evaluating http://localhost:14992/app/main.js
    Error loading http://localhost:14992/app/main.js

On closer inspection it looks like it doesn't like global for _amdLoaderGlobal, so I set it to AMDLoader. Same thing with global being set for the NLSLoaderPlugin.

It then blows up in editor.main.js with the following error:

    TypeError: define is not a function
        at eval (http://localhost:14992/jspm_packages/npm/monaco-editor@0.5.3/dev/vs/editor/editor.main.js:14:5)
        at eval (http://localhost:14992/jspm_packages/npm/monaco-editor@0.5.3/dev/vs/editor/editor.main.js:74765:6)
        at Object.eval (http://localhost:14992/jspm_packages/npm/monaco-editor@0.5.3/dev/vs/editor/editor.main.js:75181:3)
        at eval (http://localhost:14992/jspm_packages/npm/monaco-editor@0.5.3/dev/vs/editor/editor.main.js:75183:4)
        at eval (http://localhost:14992/jspm_packages/npm/monaco-editor@0.5.3/dev/vs/editor/editor.main.js:75184:3)
    Evaluating http://localhost:14992/jspm_packages/npm/monaco-editor@0.5.3/dev/vs/editor/editor.main.js
    Evaluating http://localhost:14992/jspm_packages/npm/monaco-editor@0.5.3.js
    Evaluating http://localhost:14992/app/app.component.js
    Evaluating http://localhost:14992/app/main.js
    Error loading http://localhost:14992/app/main.js

Setting the window require and define to the systemJS implementations of amdRequire and amdDefine doesn't help, but setting the define / require in the editor.main.js to use the window level objects does.

At that point, it hits a monaco is undefined error.

Edit: here's the systemJS mapping:

map:{
"monaco-editor": "npm:monaco-editor@0.5.3"
},
"npm:monaco-editor@0.5.3": {
     "fs": "github:jspm/nodelibs-fs@0.1.2",
      "os": "github:jspm/nodelibs-os@0.1.0",
      "path": "github:jspm/nodelibs-path@0.1.0",
      "process": "github:jspm/nodelibs-process@0.1.2"
},

And the requires generated:

module.exports = require("npm:monaco-editor@0.5.3/dev/vs/loader");
module.exports = require("npm:monaco-editor@0.5.3/dev/vs/editor/editor.main");

chrisber commented 8 years ago

thanks to @Matthias247 I got it working with angular rc5 with angular-cli: 1.0.0-beta.11-webpack.8 @fxck @waywaaard my additions to @Matthias247 can be found here: https://gist.github.com/chrisber/ef567098216319784c0596c5dac8e3aa

superRaytin commented 8 years ago

I have created a package named react-monaco-editor, it based on the approaches mentioned above (thanks to @Matthias247), hope it helps if someone uses React stack. Also hope it to be abandoned in the near future (after a better way appeared).

danvk commented 8 years ago

For anyone using TypeScript and React, here's the component I came up with:

/// <reference path="../node_modules/monaco-editor/monaco.d.ts" />

import * as React from 'react';

declare const require: any;

interface Props {
  value: string;
  language: string;
  onChange: (newValue: string) => any;
}

export default class MonacoEditor extends React.Component<Props, {}> {
  editor: monaco.editor.IStandaloneCodeEditor;

  render(): JSX.Element {
    return <div className='monaco-editor' ref='editor'></div>;
  }

  componentDidMount() {
    // Monaco requires the AMD module loader to be present on the page. It is not yet
    // compatible with ES6 imports. Once that happens, we can get rid of this.
    // See https://github.com/Microsoft/monaco-editor/issues/18
    (window['require'])(['vs/editor/editor.main'], () => {
      this.editor = monaco.editor.create(this.refs['editor'] as HTMLDivElement, {
        value: this.props.value,
        language: this.props.language,
        lineNumbers: false,
      });

      this.editor.onDidChangeModelContent(event => {
        this.props.onChange(this.editor.getValue());
      });
    });
  }

  componentDidUpdate(prevProps: Props) {
    if (prevProps.value !== this.props.value && this.editor) {
      this.editor.setValue(this.props.value);
    }

    if (prevProps.language !== this.props.language) {
      throw new Error('<MonacoEditor> language cannot be changed.');
    }
  }
}

bvaughn commented 8 years ago

@danvk, @Matthias247: How are you working around the edcore.main and fs errors in vs/language/typescript/lib/typescriptServices.js? The approaches you've both mentioned above still fail mid-way for me.

Sorry if this is a silly question. Kind of confused though as I've tried essentially the same thing the 2 of you mentioned but with different results.

Matthias247 commented 8 years ago

@bvaughn I had that error only when trying to use Webpack ProvidePlugin. The other approach that I describe here which uses CopyWebpackPlugin and monacos AMD loader does not produce an error.

bvaughn commented 8 years ago

Hm. I was getting the error using their AMD loader and require.config (rather than CopyWebpackPlugin). I tried CopyWebpackPlugin at one point earlier but wasn't able to get it working.

0plus1 commented 7 years ago

@Matthias247 I took your example and nicely packaged it into an ugly npm package https://github.com/0plus1/ng2-monaco-editor, it's there for myself mainly and for the community, if anybody wishes to contribute please feel free!

siovene commented 7 years ago

Regarding @Matthias247 's example, I would like to add that using this angular2 seed, it's not necessary to do any symlinking or using webpack to copy files.

You can simply import from node_modules as shown below:

    ngAfterViewInit() {
        var onGotAmdLoader = () => {
            // Load monaco
            (<any>window).require.config({ paths: { 'vs': 'node_modules/monaco-editor/min/vs' } });
            (<any>window).require(['vs/editor/editor.main'], () => {
                this.initMonaco();
            });
        };

        // Load AMD loader if necessary
        if (!(<any>window).require) {
            var loaderScript = document.createElement('script');
            loaderScript.type = 'text/javascript';
            loaderScript.src = 'node_modules/monaco-editor/min/vs/loader.js';
            loaderScript.addEventListener('load', onGotAmdLoader);
            document.body.appendChild(loaderScript);
        } else {
            onGotAmdLoader();
        }
    }

0plus1 commented 7 years ago

@siovene thanks for that, tested it on angular-cli and it doesn't work :-(

Matthias247 commented 7 years ago

Maybe the build script for this special project already copies the whole node_modules folder to the output. This would means monaco is in the correct place. Or it drops the main output file into the main folder of the project. However it's hard to tell what exactly it does since it's so big. In the general case you have to assure that monaco is available in the built project.

Matthias247 commented 7 years ago

Btw, I improved my approach since the last posting a little bit. I moved all monaco loading code into a seperate angular2 service component/class:

monacoeditorloader.ts:

declare const require: any;

export class MonacoEditorLoader {
  private _loaded = false;
  private _loadPromise: Promise<void>;

  constructor(
  ) {
    this._loadPromise = new Promise<void>(resolve => {
      // Fast path - monaco is already loaded
      if (typeof((<any>window).monaco) === 'object') {
        resolve();
        return;
      }

      var onGotAmdLoader = () => {
        // Load monaco
        (<any>window).require(['vs/editor/editor.main'], () => {
          this._loaded = true;
          resolve();
        });
      };

      // Load AMD loader if necessary
      if (!(<any>window).require) {
        var loaderScript = document.createElement('script');
        loaderScript.type = 'text/javascript';
        loaderScript.src = 'vs/loader.js';
        loaderScript.addEventListener('load', onGotAmdLoader);
        document.body.appendChild(loaderScript);
      } else {
        onGotAmdLoader();
      }
    });
  }

  get monacoLoaded() {
    return this._loaded;
  }

  // Returns promise that will be fulfilled when monaco is available
  waitForMonaco(): Promise<void> {
    return this._loadPromise;
  }
}

I expose that through an angular2 module, which declares this as a provider amongst some other common services:

@NgModule({
  declarations: [],
  imports: [
    BrowserModule,
    HttpModule,
  ],
  providers: [
    ...
    MonacoEditorLoader,
  ],
})
export class ServicesModule {
}

In all components that want to utilize monaco later on I get the loader through DI, wait for the load to complete and create monaco editors. The advantage of this approach is that loading is separated from utilizing Monaco and that it's easy to use Monaco in multiple components without big code duplication. The only drawback is that loading is still asynchronous - the editor won't be immediatly there after ngAfterViewInit. It's sometimes visible that the editor pops up later. But for me that's not a real problem.

@Component({
  selector: 'editor',
  template: `
    <div #editor class="editor-container" style="width: 1000px; height: 1000px">
  `,
})
export class Editor implements OnInit, AfterViewInit, OnDestroy {
  @ViewChild('editor') editorRef: ElementRef;
  private _disposed = false;
  private _editor: monaco.editor.IStandaloneCodeEditor;

  constructor(
    private _monacoLoader: MonacoEditorLoader
  ) {
  }

  ngAfterViewInit() {
    // Wait until monaco editor is available
    this._monacoLoader.waitForMonaco().then(() => {
      // Need to check if the view has already been destroyed before Monaco was loaded
      if (this._disposed) return;
      this.initMonaco();
    });
  }

  ngOnDestroy() {
    if (this._disposed) return;
    this._disposed = true;

    // Close possibly loaded editor component
    if (this._editor) this._editor.dispose();
    this._editor= null;
  }

  initMonaco() {
    this._editor = monaco.editor.create(this.editorRef.nativeElement, {
      value: '',
      language: 'javascript',
    });
  }
}

In my real project I have actually even a step in between, as I want to add some custom functionality (syntax highlighting, autocompletion, ...) to Monaco. I put this into another service which uses the MonacoEditorLoader to load Monaco and on Completion it adds the new functionality. My view components then wait on the service in between to finish loading instead of waiting directly on the MonacoEditorLoader. But that really depends on what you exactly want to achieve.

mbana commented 7 years ago

i think the html5 imports* is probably the most simple approach at the moment:

the server of monaco will need to enable http2 to get decent performance (based on experience).
html5 imports is a decent-enough module system. i was suprised when i discovered what one could do with it.
in Polymer, one would use a similar tool to bundle web components (think AMD modules), see https://github.com/Polymer/vulcanize. however, this is going to become obselete due to http2.
build process becomes much simpler.

all in all, everything becomes much simpler.

@alexandrudima, have you had a chance to look at this?

as an example, when editor.main has finished loading, the script below executes.

<script src="vs/editor/editor.main"></script>
<script>
var editor = monaco.editor.create(document.getElementById('container'), {
    value: [
        'function x() {',
        '\tconsole.log("Hello world!");',
        '}'
    ].join('\n'),
    language: 'javascript'
});
</script>

http://webcomponents.org/articles/introduction-to-html-imports/

mattgodbolt commented 7 years ago

I'm encountering similar problems when trying to use require.js and r.js. A normal build works locally, but trying to generate a single bundle with r.js yields errors of the form:

Tracing dependencies for: main
Error: ENOENT: no such file or directory, open '/home/mgodbolt/personal/dev/compiler-explorer/out/dist/ext/monaco-editor/dev/vs/editor/edcore.main.js'
In module tree:
    main
      hub
        editor
          colour
            monaco
              vs/editor/editor.main

and edcore.main.js is nowhere to be found in the npm package. Ideas welcome on how to fix this (would love to switch to monaco, but can't until I can deploy!)

Kaijun commented 7 years ago

any updates?

kube commented 7 years ago

I'm using Electron with Webpack, and integrating Monaco is really a mess.

I cannot load Monaco by appending the loader.js script, as the loader won't recognize in which context it is, and I get too much errors.

And when loading using Webpack require directly

require.ensure(['vs/editor/editor.main'], () => {
  require('vs/editor/editor.main')
  // ...
})

It cannot findedcore.main:

Module not found: Error: Can't resolve 'vs/editor/edcore.main' in 'node_modules/monaco-editor/dev/vs/editor'

kube commented 7 years ago

@alexandrudima

From what I see in the gulpfile edcore.main is simply an alias to editor.main. But I don't see any define of it.

https://github.com/Microsoft/monaco-editor/blob/master/gulpfile.js#L100

I don't really understand what this does, and I never used AMD modules before so it's a little bit obscure for me.

I really need to integrate Monaco in my setup with Webpack and Electron, and I'm pretty sure it's not much to modify to make it work.

Could you explain to us what this addPluginContribs function really does, and what really is edcore.main?

Monaco and VSCode are awesome projects, and it's demoralizing to see that we can not use it in our Webpack projects.

cloverich commented 7 years ago

@kube I used the below based on what alexandru posted in another issue (which I seem to have misplaced).

index.html in an Electron / webpack project:

    <script>
        // require node modules before loader.js comes in
        const path = require('path');
        const nodeRequire = global.require;
    </script>
    <script src="../node_modules/monaco-editor/min/vs/loader.js"></script>
    <script>
        const amdRequire = global.require;
        global.require = nodeRequire;

        function uriFromPath(_path) {
            var pathName = path.resolve(_path).replace(/\\/g, '/');
            if (pathName.length > 0 && pathName.charAt(0) !== '/') {
                pathName = '/' + pathName;
            }
            return encodeURI('file://' + pathName);
        }

        amdRequire.config({
            baseUrl: uriFromPath(path.join(__dirname, '../node_modules/monaco-editor/min'))
        });

        // workaround monaco-css not understanding the environment
        self.module = undefined;

        // workaround monaco-typescript not understanding the environment
        self.process.browser = true;

        amdRequire(['vs/editor/editor.main'], function() {

        });
    </script>

Also note that when running the project in Electron, you don't actually need webpack -- you can require all of your source directly into the browser. Hope this helps!

kube commented 7 years ago

@cloverich

As I said:

"I cannot load Monaco by appending the loader.js script"

And it's exactly this solution I used, and it does not work well when using Webpack (Webpack proxifies all require calls, and it's not a good idea to start hacking the require function).

"Also note that when running the project in Electron, you don't actually need webpack"

I know what is Electron, and using it with Webpack permits to have Hot Reload, a transpilation pipeline easier to setup than with a tool like Gulp, and all the stuff Webpack and its DevServer permit.

cloverich commented 7 years ago

@kube that is the solution I used when I was using webpack. Can you share the your code, or a minimum setup with the issue?

icfantv commented 7 years ago

@Matthias247 Hey, I'm interested in incorporating your findings into our project. Is what you have here the latest version? Also, I'm interested in seeing the custom functionality bit. Thanks.

Matthias247 commented 7 years ago

@icfantv Hey, yes, that's basically still my latest version. In the real code of the component which uses the editors I had some more code around the lifecycle hooks which allow for safe destruction and recreation of editor components. I added a little bit more around that in the example above.

Unfortunately can't show more about the remaining code since it contains mostly proprietary features. But the idea is simply that the intermediate component which adds custom functionality waits for the basic monaco loader to be ready, calls Monaco APIs to add custom syntax highlighting, autocompletions, etc. and then dispatches a ready event itself (or completes a promise). The component which wants to use a Monaco editor in the end then waits for this ready event instead of the _monacoLoader.waitForMonaco() event. E.g. it could be _monacoWithExtensionsLoader.waitForMonaco().

icfantv commented 7 years ago

@Matthias247 got it and looks easy enough. Thanks again for posting this.

P-de-Jong commented 7 years ago

So i've managed to get it working with the Angular CLI. In you angular-cli.json put this in the assets:

"assets": [
       "assets",
       "favicon.ico",
       {
         "glob": "**/*",
         "input": "../node_modules/monaco-editor/min/vs",
         "output": "./assets/monaco/vs/"
       }
],

This will serve the monaco folder in the assets folder. When building the application it will also copy the files to the dist/assets folder. Works great! 👍

Then use the code provided by @Matthias247 over here, but replace the 'loadPromise' in the monacoeditorloader.ts with the following:

this._loadPromise = new Promise<void>(resolve => {
            // Fast path - monaco is already loaded
            if (typeof ((<any>window).monaco) === 'object') {
                resolve();
                return;
            }

            const onGotAmdLoader = () => {
                // Load monaco
                (<any>window).require.config({ paths: { 'vs': '/assets/monaco/vs' } });
                (<any>window).require(['vs/editor/editor.main'], () => {
                    this._loaded = true;
                    resolve();
                });
            };

            // Load AMD loader if necessary
            if (!(<any>window).require) {
                const loaderScript = document.createElement('script');
                loaderScript.type = 'text/javascript';
                loaderScript.src = '/assets/monaco/vs/loader.js';
                loaderScript.addEventListener('load', onGotAmdLoader);
                document.body.appendChild(loaderScript);
            } else {
                onGotAmdLoader();
            }
        });

FrankFang commented 7 years ago

What a pity. No webpack support.

leolorenzoluis commented 7 years ago

Hello everyone,

Thanks to @Matthias247. I created a simple library for Angular to make sure that Monaco editor is ready without using timeouts or then with Promises. I did not want to clutter my components with those functions. Please check it out.

Introducing 🎉🎉🎉 Angular Monaco Editor Loader 🎉🎉🎉

https://github.com/leolorenzoluis/xyz.MonacoEditorLoader

All I have to do now is

<div *loadMonacoEditor id="container"></div>

TheLarkInn commented 7 years ago

Hi there!!! Sean (from webpack). I'd love to discuss the possibility of shipping Monaco using the Harmony Module format (esm). What's the best medium to propose this?

alexdima commented 7 years ago

@TheLarkInn

I'm not intimately familiar with webpack, and I've only tangentially followed the ES Harmony module proposal, but I am quite knowledgeable in AMD (having written our AMD loader, bundler, etc.). I can describe what we're doing and why we're doing it and hopefully we can use our collective brain power to come to a good solution.

Some of the requirements are probably quite general of any large software project, some of them are unique...

Basic considerations

the monaco-editor node module comes together by "stitching" together the monaco-editor-core node module and several monaco-* plugins (also distributed via npm) that add various language supports. Their location is described here. This makes it possible, by forking this repository (and tweaking the metadata.js root file), for someone to create a monaco-editor distribution which contains the html plugin, but not the json plugin, etc. Or someone can package a monaco-editor distribution with an additional plugin that they author.
for the purpose of better supporting webpack, we can change the monaco-* plugins shape more or less freely (changing them only impacts the monaco editor), so we don't need to think too deeply about them, except for the lazy loading point below.
the monaco-editor-core node module, (a language agnostic editor), gets shipped straight from VS Code's sources. We must maintain single source with VS Code, otherwise all the good work we're doing for the VS Code editor would probably never make it to the monaco editor.
- VS Code is the editor's "home". The monaco-editor-core simply shims many services the editor code needs such that it can be hosted in a standalone manner. e.g. IProgressService, IMessageService, IKeybindingService, etc
- we want to keep our CSS next to the code that needs it. e.g.
- hover.ts requires `vs/css!./hover'
- this indicates that the factory method of hover.ts should be invoked only after the CSS in hover.css has been loaded.
- we want i18n to be painless for developers. e.g.
- we import nls from vs/nls
- to define a translatable string, we make a simple call in the source code to nls.localize where we define an identifier (that disappears at runtime) and the English value for the string. Sometimes, we can even provide more metadata with explanations for translators if we make the call with a different shape. e.g. : simple, with comments intended for translators
- we have an entire translation pipeline set up for VS Code from which the Monaco Editor benefits. nls.localize calls can all be analyzed statically and strings to be translated are extracted. They then get pushed to a format Transifex likes and then translations can be pulled back from it and get converted to .json files in our repository.
- when we compile with a "release" target in mind (i.e. not just for running out of sources), all the code that required vs/nls gets patched to signal the caller module it e.g. a require of vs/nls from vs/editor/contrib/hover/browser/hover.ts gets rewritten to vs/nls!vs/editor/contrib/hover/browser/hover, all the English strings are extracted to a separate file hover.nls.js and all the nls.localize calls are rewritten to pass in the numeric index of the string in a file, to give a nice array cost (O(1)) of using a localized string.
- at bundle time, we start with vs/editor/editor.main and ship only the subset of modules that are needed (referenced) for the monaco editor.
- nls: the vs/nls plugin participates in the bundling phase and concatenates all the nls strings in a single file, and consolidates dependencies. e.g. vs/nls!vs/editor/contrib/hover/browser/hover is defined as define('vs/nls!vs/editor/contrib/hover/browser/hover', ['vs/nls', 'vs/nls!vs/editor/editor.main']), function(nls, data) { return nls.create("vs/editor/contrib/hover/browser/hover", data); });. so all of the nls dependencies end up redirecting to vs/nls!vs/editor/editor.main, and the plugin generates various nls.*.js bundles for the various languages we ship (e.g. German, Chinese, etc.)
- css: the vs/css plugin also participates in the bundling phase and concatenates all the CSS, taking of course into consideration that the relative paths to images have changed and inlines all of the images below a certain size (we have lots of small svg icons).
- the bundler does some simple transformations to reduce code size, such as eliminating duplicated TS helper definitions and hoisting up all the AMD module names such that they don't get repeated again and again and again.
- finally, "because life", we shim our octicons usage since we want those in VS Code, but their size cost is not paying off for the monaco editor.

Requirements

Even after all of this stuff has happened and we have extracted the precise subset of source code, nls strings, CSS, and images that the monaco editor needs out of VS Code, there are still some special requirements we have.

We want to use web workers to have a never blocking UI. e.g. if you use the monaco diff editor, the diff is always computed in a web worker. if you use JS/TS/etc, the ASTs and analysis and heavy lifting of computing completions, etc. happens on a web worker.

We want to lazy load languages. e.g. if you instantiate an editor model with the language css, we want to dynamically load only the CSS specific code. i.e. on the main thread, we load the CSS colorizer and the CSS part of the plugin that registers the CSS language smarts provider; the CSS plugin makes use of some straight forward API to create a web worker and load its specific code in the new web worker. The basics of a web worker that can do buffer sync is at vs/base/worker/workerMain.js.

If, later, you instantiate an editor model with the language html, etc, the same happens for html.

Integrator burden

we try to hide most of this complexity to the point where an integrator "simply" needs to ship a bit-to-bit copy of the files we distribute to production, and the lazy loading of programming languages and the configurable loading of natural languages aspect brings in the runtime dependency towards an AMD loader.

If the integrator has their own AMD compliant loader (i.e. that supports plugins), we will run with that. Otherwise, integrators can use the AMD loader we supply at vs/loader.js.

e.g. a simple-case integrator can get away with:

require.config({ paths: { 'vs': '../path/to/vs/folder' } });
require(['vs/editor/editor.main'], function() { /* editor loaded here */ })

A fancier integrator that is concerned about the loading waterfall can load the editor synchronously by doing something like. e.g.:

<link rel="stylesheet" data-name="vs/editor/editor.main" href="path/to/vs/editor/editor.main.css">
<script>var require = { paths: { 'vs': 'path/to/vs' } };</script>
<script src="path/to/vs/loader.js"></script>
<script src="path/to/vs/editor/editor.main.nls.js"></script>
<script src="path/to/vs/editor/editor.main.js"></script>
<script>/* editor loaded here */</script>

An integrator that wants to load a different natural language can either load editor.main.nls.de.js or configure the nls plugin. e.g.

Conclusions

My problem is I'm not really sure what the requirement is here and what the desire is.

Do integrators want to bundle the monaco editor through their own webpack config

Do we then need to ship 1000s of files, some of them not needed (i.e. files only used in VS Code)
Do the integrators want to deal with patching our nls.localize calls and correlating those with translated strings and creating nls bundle files?
Do the integrators want to deal with concatenating our 100s of CSS files and inlining small svg icons?
Do the integrators want to deal with our N entry points: vs/editor/editor.main.js, vs/base/worker/workerMain.js, vs/basic-languages/src/XXXX.js, vs/languages/json/{jsonMode,jsonWorker}.js, do the integrators want to know that the TS/JS plugin consists of 3 files: a mode.js, a worker.js and typescriptServices.js that is loaded both on the main thread and in the worker -- it is loaded on the main thread for colorization purposes and on the worker thread for language services purposes.

Do integrators want to load the monaco editor with webpack

but... this should just work, right? since webpack supports loading AMD code.

I apologise if I'm missing the point, supporting the Harmony module format might be as easy as us replacing our calls to define('moduleId', ['req1', 'req2', ... ]) with module "moduleId" { import X from 'req1' ... } and our lazy loading from require(['lazyCode'], function() { ... }) with Loader.load calls, but there still remains the challenge of what vs/css and vs/nls should map to. Shipping our code in such a shape would also mean the editor source code is not loadable in IE11 without a loader that ... loads the code via xhr and uses eval / regexes to discover those dependencies ?

I'm also happy to do more reading on the subject if you know of a good article explaining what's expected from us to be ES Harmony module compliant in what we ship. Or ... if you have any ideas, I'd love to hear them.

otakustay commented 7 years ago

In CodeMirror we can bundle it with webpack by hacking a little of it's public overridable APIs, heres what we tried and succeed:

import {noop} from 'lodash';
import CodeMirror from 'codemirror/lib/codemirror';

import 'codemirror/mode/meta';
import 'codemirror/mode/brainfuck/brainfuck';
import 'codemirror/mode/clike/clike';
import 'codemirror/mode/cmake/cmake';
import 'codemirror/mode/coffeescript/coffeescript';
import 'codemirror/mode/commonlisp/commonlisp';
import 'codemirror/mode/css/css';
import 'codemirror/mode/d/d';
import 'codemirror/mode/diff/diff';
import 'codemirror/mode/eiffel/eiffel';
import 'codemirror/mode/fortran/fortran';
import 'codemirror/mode/gherkin/gherkin';
import 'codemirror/mode/go/go';
import 'codemirror/mode/groovy/groovy';
import 'codemirror/mode/htmlembedded/htmlembedded';
import 'codemirror/mode/htmlmixed/htmlmixed';
import 'codemirror/mode/idl/idl';
import 'codemirror/mode/jade/jade';
import 'codemirror/mode/javascript/javascript';
import 'codemirror/mode/jsx/jsx';
import 'codemirror/mode/julia/julia';
import 'codemirror/mode/lua/lua';
import 'codemirror/mode/markdown/markdown';
import 'codemirror/mode/mathematica/mathematica';
import 'codemirror/mode/mllike/mllike';
import 'codemirror/mode/nsis/nsis';
import 'codemirror/mode/perl/perl';
import 'codemirror/mode/php/php';
import 'codemirror/mode/powershell/powershell';
import 'codemirror/mode/properties/properties';
import 'codemirror/mode/protobuf/protobuf';
import 'codemirror/mode/puppet/puppet';
import 'codemirror/mode/python/python';
import 'codemirror/mode/q/q';
import 'codemirror/mode/rpm/rpm';
import 'codemirror/mode/rst/rst';
import 'codemirror/mode/ruby/ruby';
import 'codemirror/mode/rust/rust';
import 'codemirror/mode/sass/sass';
import 'codemirror/mode/scheme/scheme';
import 'codemirror/mode/shell/shell';
import 'codemirror/mode/smarty/smarty';
import 'codemirror/mode/soy/soy';
import 'codemirror/mode/sql/sql';
import 'codemirror/mode/stex/stex';
import 'codemirror/mode/swift/swift';
import 'codemirror/mode/tcl/tcl';
import 'codemirror/mode/toml/toml';
import 'codemirror/mode/troff/troff';
import 'codemirror/mode/ttcn-cfg/ttcn-cfg';
import 'codemirror/mode/vb/vb';
import 'codemirror/mode/vbscript/vbscript';
import 'codemirror/mode/verilog/verilog';
import 'codemirror/mode/xml/xml';
import 'codemirror/mode/yaml/yaml';

import 'codemirror/addon/search/search';
import 'codemirror/addon/search/searchcursor';
import 'codemirror/addon/search/jump-to-line';
import 'codemirror/addon/search/matchesonscrollbar';
import 'codemirror/addon/search/match-highlighter';
import 'codemirror/addon/scroll/annotatescrollbar';
import 'codemirror/addon/scroll/simplescrollbars';
import 'codemirror/addon/dialog/dialog';
import 'codemirror/addon/selection/active-line';
import 'codemirror/addon/display/fullscreen';
import 'codemirror/addon/display/autorefresh';
import 'codemirror/addon/edit/trailingspace';
import 'codemirror/addon/mode/overlay';

import 'codemirror/lib/codemirror.css';
import 'codemirror/addon/dialog/dialog.css';

import './CodeMirror.global.less';

CodeMirror.requireMode = noop;
CodeMirror.autoLoadMode = noop;

export default CodeMirror;

These code is simple:

importing CodeMirror from npm packages
importing modes (language syntax highlighting) and addons (plugins) on demand
Override the requireMode and autoLoadMode static methods to noop, this disables the "loader" shipped by CodeMirror (which originally supports both CJS and AMD)

After override requireMode and autoLoadMode it's not possible to load language modes or addons on demand, so when dynamic load is required, we can just leave these two methods and do a copy/paste work, in this way CodeMirror works similar to how current monaco-editor behaves

Bundle with webpack is very useful for applications in a predictable and fast network environment

OneCyrus commented 7 years ago

has anyone tried to auto convert it with tools like amd-to-es6?

rebornix commented 7 years ago

@alexandrudima already had a thorough check list above. I played with webpack these few days and synced with @TheLarkInn . Here are my findings, correct me if I made anything wrong.

Integrate with Webpack: Load

There are already a few solutions above about how to make Monaco Editor work with Webpack, the key point is

Ask webpack not to resolve Monaco related modules, e.g. vs/editor/editor.main.
Copy Monaco bits to output folder, you can use CopyWebpackPlugin.
Use a amd loader to load Monaco Editor and corresponding plugins.

We can even put amd loader into a closure to avoid conflict with other module system.

Integrate with Webpack: Bundle

Ppl may want to bundle Monaco editor with Webpack. Webpack will try to resolve modules when bundling Monaco modules, to make it compatible:

For vs/css!, we can use CSS loader to bundle CSS the same way as our AMD CSS Plugin. One catch here is we don't put file extension to CSS file when referencing, we can use NormalModuleReplacementPlugin to mitigate this problem.
For vs/nls!, we need to write a custom loader for NLS.

However, if we directly use webpack to bundle our npm package, we'll see following errors/warnings like below

Cannot resolve module vs/editor/edcore.main
define cannot be used indirect
Critical dependency: require function is used in a way in which dependencies cannot be statically extracted
Critical dependency: the request of a dependency is an expression

This is due to Webpack's module resolution mechanism. It tries to resolve modules statically so code like

var module = languageDefinitions[languageId].module;
return new _monaco.Promise(function (c, e, p) {
      require([module], function (mod) {         
      }, e);
});

won't work as webpack doesn't know to evaluate module. This explains

Cannot resolve module vs/editor/edcore.main

we didn't use string literal directly when defining vs/editor/edcore.main in the bundled main entry editor.main.js, it confuses webpack. Workarounds are using string literal when bunding Monaco or shipping all files and then webpack can resolve modules by file paths/names.

Critical dependency: require function is used in a way in which dependencies cannot be statically extracted Critical dependency: the request of a dependency is an expression

If those modules already exist in the component, we can leverage https://webpack.github.io/docs/context.html#dynamic-requires to solve the problem. We still use above code snippet as example

https://github.com/Microsoft/monaco-languages/blob/master/src/monaco.contribution.ts#L26

var module = languageDefinitions[languageId].module;
return new _monaco.Promise(function (c, e, p) {
      require([module], function (mod) {         
      }, e);
});

As we know all language modules are in the same folder, so changing it to

var module = languageDefinitions[languageId].module;
return new _monaco.Promise(function (c, e, p) {
      require(['./' + module], function (mod) {         
      }, e);
});

and then webpack will create a context module, this module contains references to all modules in that directory that matches a certain pattern.

There are a few other places (in the core and all builtin extensions) that we need to apply dynamic require.

Open Items

I can already bundle Monaco-Editor-Core + Basic Languages with Webpack but didn't succeed bundling all extensions. There are still several questions I don't have answer yet

Can we define multiple AMD modules in a single file and tell Webpack to resolve modules from that file?
Is it possible to tell webpack skip some require function all when webpack_require fails? There are some places that the modules we try to require are real dynamic (they don't exist yet when bundling), falling back to a simple require and then the module can be loaded correctly on runtime. https://github.com/webpack/webpack/issues/4545

If above two problems are solved someway, then the left work is tweaking core and extensions, creating entry points for both main thread and web worker, even though it still requires a lot code change but still doable.

TheLarkInn commented 7 years ago

Can we define multiple AMD modules in a single file and tell Webpack to resolve modules from that file?

I'm not sure what you mean, could you provide an example.

Is it possible to tell webpack skip some require function all when webpack_require fails? There are some places that the modules we try to require are real dynamic (they don't exist yet when bundling), falling back to a simple require and then the module can be loaded correctly on runtime. webpack/webpack#4545

Yes, you can! Just as the NormalModuleReplacement allows one to "hijack and modify" the request from a NormalModule, the ContextReplacementPlugin will allow one to define a custom context, or filter an existing ContextModule request. You can find more information in this thread.

rebornix commented 7 years ago

@TheLarkInn thanks for the answer of require fallback. About my first question, it's about resolving modules in bundled files. Let's say I have a bundled file (which might be from node modules), it contains several modules

a.js

define('b', [], function () {
}

define('a', ['./b'], function () {
}

and we try to reference these two modules in entry file index.js.

define('index', ['./a', 'b'], function () {
}

a is accessible as the file a.js is in the correct folder. While webpack resolving module a, as module b is in the same file and already defined, webpack knows how to resolve a's dependencies.

My question is, is there any way to reference module b in entry files (index.js)? Moreover how can we require module a and b at runtime since webpack bundles them and probably replace the module name with absolute path.

otakustay commented 7 years ago

is there any way to reference module b in entry files (index.js)?

As webpack ks a module bundler, there is no separated module in a bundle by default, you can make modules accessible by exporting them in your entry, or to configure webpack to export some modules, in both solutions the exports must be explicitly defined

how can we require module a and b at runtime

Webpack can bundle a CommonJS/AMD library so you can reference the module using CommonJS require or an AMD loader, but your code is probably:


define('myApp', ['monaco'], function (monaco) {
    const a = monaco.a;
    const b = monaco.b;

    // ...
});

ProTip commented 7 years ago

I hope I'm not noising this thread up, but it has by far been the most useful resource in getting monaco working with my setup. I'm utilizing Monaco in an electron app and was having a little trouble with the loader @Matthias247 posted. Here is my working version based on a mashup of the above example and the official electron guidance:

declare const global: any
declare const require: any

const anySelf: any = self

const nodeRequire = global.require
let amdLoader

export default class MonacoEditorLoader {
    private _loaded = false
    private _loadPromise: Promise<void>

    // Returns promise that will be fulfilled when monaco is available
    waitForMonaco(): Promise<void> {
        if (this._loadPromise) {
            return this._loadPromise
        }
        this._loadPromise = new Promise<void>(resolve => {
            if (typeof((<any>window).monaco) === 'object') {
                resolve()
                return
            }

            var onGotAmdLoader = () => {
                amdLoader = global.require
                global.require = nodeRequire

                anySelf.module = undefined
                anySelf.process.browser = true

                amdLoader(['vs/editor/editor.main'], () => {
                    this._loaded = true
                    resolve()
                })
            }

            let loaderScript = document.createElement('script')
            loaderScript.type = 'text/javascript'
            loaderScript.src = 'vs/loader.js'
            loaderScript.addEventListener('load', onGotAmdLoader)
            document.body.appendChild(loaderScript)
        })
        return this._loadPromise
    }
}

I have copied vs into my build output directory to make it work.

It would be great if we could import and have everything JustWork™. My mind wandered to (BR)ACE and CodeMirror at first when hitting the initial blocks, but stuck with it because how much I love vscode and TypeScript. To live its life to the fullest!!! outside vscode it may need to meet developer's modern bundling/importing/building expectations for low friction adoption. Not sure if that means vscode needs to dogfood it as an external yet internal package or what, but the project appears to be sufficiently complex enough I'll leave that to the experts ;)

microsoft / monaco-editor