Suggestion: multi-file external modules

[RyanCavanaugh]

Support compiling multiple input .ts files into one external module.

Need to determine exactly how the module boundaries are defined when doing this.

[basarat]

:+1:

[vvakame]

:+1:

[reverofevil]

I'm highly interested in this issue getting closed. I don't like an idea of concatenating *.ts files in my source directory with a build script.

What do you mean by "module boundaries"?

[basarat]

What do you mean by "module boundaries"?

Given:

a.ts:

export class A{}

b.ts:

export class B{}

---

What will the .d.ts will be?

---

This 1:

declare module 'foo'{
     class A{};
     class B{};
}

or 2

declare module 'foo/a'{
     class A{};
}
declare module 'foo/b'{
     class B{};
}

---

My vote : 1

[reverofevil]

Why should it create a module for each class at all, if we're explicitly making one module file?

My vote goes for the first option.

[basarat]

Support compiling multiple input .ts files into one external module.

@RyanCavanaugh can you append the following as well

Support compiling multiple input .ts files into one external module. Support generating a definition file .d.ts for such an external module.

If you want I can create a separate issue for that.

[charlessolar]

To add my 2 cents, I agree I would like to see 1, but I know that can be a challenge if for example 2 different files contain the same class or variable name. I am hoping, that like the CommonJS namespace merging you already do we would simply get a 'multiple declaration' exception.

But either 1 or 2 I would like to see this implemented someway because my application uses requirejs extensively to deliver SPA functionality with durandal.

[MrJul]

I'd like to see 1) too since our current desired use case is to use one file per class/interface/enum and one folder per module.

While I think it shouldn't be too hard too implement inside the compiler, the tooling is harder, notably in VS: how do you determine which files compose a module?

Personally, what I would really like to see to solve this problem is a mix between internal and external module syntax at the language level. Something along the lines of:

// A.ts
external module "library" {
  class A { }
}

// B.ts
external module "library" {
  class B { }
}

Both files would be compiled into one module file library.js. Each file can only declare at most one external module, and cannot have any other top-level declarations if there's an external module declared.

Another benefit of this approach is that it would no longer be magic that a file becomes an external module (with its contents no longer in global scope) as soon as there's an import or export since there's the possibility of explicitly defining that a module is external. That's what most people I've seen working with TS and external modules are struggling with when introduced with the concept. (Still, the old way would still work, for compatibility reasons and implicit single-file external module).

[kevinbarabash]

My vote is for option 1 as well.

In terms of determining which files are part of a module, maybe we could use reference paths... something like this:

// library.ts
/// <reference path="A.ts"/>
/// <reference path="B.ts"/>

export = library;

If you compile this with tsc --out library.js library.ts --module commonjs --declaration I would expect there to be line at the end of library.js with module.exports = library which there isn't. Adding this line isn't a huge deal, but it would be nice if it were done automatically.

[KeithWoods]

I say my vote would be for 1 too.

If you're working in a large org or team and want to develop reusable packages as part of a large system (i.e. any large enterprise app) then you'll hit this problem.

You'd be be using external modules as you'll be relying on an external loader (unlike internal modules that assume it's all loaded already). Each module will defer to the module loader to find the correct script not by an assumption it's already loaded (this just referring to it's module object) .

The separate modules need a single .d.ts so it can be consumed by dependencies at compile time. Currently having (potentially hundreds) of single .d.ts for an external module package is useless.

We've got our system running using dts-bundle which scrapes all the single files into a single .d.ts (same approach as outlined in #17), but there are other teams here stumped by the complexity required to get this running, there off building monolithic internal module applications that can't be split out into separate packages.

related issue #1236

[xealot]

Everyone in this thread is voting for 1, so I imagine it's just me being slow to understand how it would work. With option 1 you're changing how things are actually defined if when using external modules aren't you?

If my definition for class A lives in foo/a.ts I would need to import foo/a to get access to the scope that A is defined in. If the d.ts module rewrites it to foo/A wouldn't it break the imports? Additionally, if I made the example a little more silly but still completely valid:

a.ts:

export class A{}

b.ts:

export class A{}

Would the output be:

declare module 'foo'{
     class A{};
     class A{};
}

Additionally, if 2 was produced:

declare module 'foo/a'{
     class A{};
}
declare module 'foo/b'{
     class A{};
}

How would the compiler know where the base path was? When you import this project into other projects the module names should be from and inclusive of the project root correct? So in another project the modules would really need to be defined as:

declare module 'project/foo/b'{
     class A{};
}

I imagine this is going to boil down to the addition of a --include or --library option perhaps? I'm not sure what the solution is, but our biggest stumbling block right now is trying to use typescript modules in other code effectively without shipping the entire raw source. Internal modules work for a while, but if you ever want your project to be isomorphic (node & browser) or traceable using something like browserify external modules are a must.

[kevinbarabash]

@xealot Are you suggesting that the folder structure defines modules? I'd rather module definitions be more explicit, that's why I was suggesting having a main library.ts file that contains references to the things you want in your module. The nice thing about having a separate file is that you could potential have multiple version of the library.ts which have different features.

[charlessolar]

Well currently typescript is using folder structure to import modules, at least for AMD modules it does. @xealot and @MrJul makes a good point that merging definitions like 1 would make it hard on tooling (intelisense, etc)

I agree with @MrJul 's solution by adding a new 'external module' keyword would make the most sense to me and be nice on tooling.

[xealot]

@kevinb7 I'm not suggesting that folder structure defines modules so much as pointing out that's already how things work. With the single exception of Typescript's internal module system, external TS modules, CommonJS, AMD and the upcoming ES6 standard base modules at files and their locations in the path if I'm not mistaken.

[kevinbarabash]

Time for me to go re-read the spec. :sweat:

[park9140]

I've been working with a large project for a while now, we are attempting to break down our project into multiple smaller libraries now and have run into problems around this topic.

We have come up with two possible paths this could go.

First, make our smaller libraries into internal modules, then wrap them manually into an external module.
This seems like a perfectly valid use case but requires some level of mapping file to show what portions of the internal module should be exported.

Second, keep our existing external module setup, one file for each class/interface/function, each with a single default export ('export = itemName'). The issue with this path ends up being the same as the internal module path. We still need to create a 'library' file that imports the public types and exports them.

In both of these cases, one simple problem blocks implementation. The compiler does not let you export a type declared in a different external module as part of your module.

To that end I would suggest a third option and or extension to option 1

As per the es6 exports specification http://people.mozilla.org/~jorendorff/es6-draft.html#sec-exports export ExportClause FromClause

We would then do a.ts

export default interface A{} //current form export = class A{}

b.ts

import default as A from 'a'
export default class B implements A{} 

library.ts

export {default as A} from 'a'
export {default as B} from 'b'

this should then compile a library.d.ts file like this

declare interface A {}
declare class B implements A {}

And should throw a compilation error if you don't export A from your module since B implements A

This probably means #1215 has to be finished first.

I disagree with making the use of folder structure, and automatic exports, since most of classes, functions, and interfaces used internally to a library should not be exposed unless the library explicitly needs them to be shared.

[jbondc]

Like the suggestion of @park9140. Similarly, I've been using the following pattern:

Add a comment on internal modules: a.ts

module MyLib {
   export class A {}
}
// export default MyLib.A

b.ts

module MyLib {
   export class B {}
}
// export default MyLib.B

lib.ts

module "mylib" {
   // import default as A from "./a";
   // import default as B from "./b";
}

With some hackery in node, internal modules are evaluated in an 'internal sandbox' and we return the default export in the sandbox: e.g.

vm.runInContext(code, sandBoxInternal, "a.ts")
return sandBoxInternal[sandBoxInternal.export.default] || null

'external' modules still use the regular require(). At build time (tsc lib.ts), you can concatenate the 'internal' modules (a.ts, b.ts) inside your external module.

[nycdotnet]

I hate being in a position where I am feeling pain but I don't have a suggestion for how to fix it - I just want it to go away. When I use TypeScript external modules for something non-trivial, I'm in pain.

Some smart people on this thread have proposed fixes - if @park9140 's aligns with ES6, that sounds awesome to me.

[kevinbarabash]

One library I created I wanted to have things in separate files and then export each modules from a common namespace... I ended up doing this in lib.ts:

export import A = require("a");
export import B = require("b");

[park9140]

@kevinb7, Yeah I did that in .8 or something, but in .9.5 they disallowed exporting imports :( this made me very sad :(

[RyanCavanaugh]

export import ... is allowed.

[park9140]

Nevermind @RyanCavanaugh is right I had it backward it was disabled in .8.3 and enabled in .9.1 based on the check I just made.

[csnover]

@xealot I agree that “option 1” doesn’t make sense. In the context of how all the module loaders work today and how the default Web loader (not ES6 any more!) is planned to work, external modules aren’t combined that way for consumption. I think you’re also right there needs to be a way to change what default library/libraries are loaded when compiling, as lib.d.ts doesn’t set up an appropriate environment for a lot of development.

[marcuswhit]

+1 This is a huge issue for large modular projects, only currently solvable with a lot of plumbing and hacks. Is this anywhere on the TS team's radar?

[csnover]

@marcuswhit I am not on the TS team but I am working on a prototype using the new language services in TS 1.4.

[fdecampredon]

For the purpose of generating a declaration for an entire commonjs module composed of multiple one I created a little script that does the job, hope it can help

[csnover]

Per my previous comment, a prototype is available at https://github.com/sitepen/dts-generator for bundling multiple modules from a package together into a single d.ts for distribution using the language services. There are a few known issues, particularly in the requirements for compiling and running (the services in 1.4 don’t expose some important APIs), but I didn’t correct those yet since it’s just intended to be a proof of concept that the TS team can look at for this proposal. However, if it is useful to you, please feel free to use it and submit patches.

[mtraynham]

@csnover I tried your generator. Seems to be very similar to https://github.com/TypeStrong/dts-bundle (might be some obvious differences that I missed).

The TypeScript compiler seems to work correctly for both of these projects. Webstorm on the other hand, not so much. It doesn't resolve class properties on import/export of external modules. I havn't tried this with Visual Studio.

Driver:

import module = require('test-module');
var x: module.Foo = new module.Foo();
console.log(x.name);

Doesn't work:

declare module 'test-module/Foo' {
    class Foo {
        public name: string;
    }
    export = Foo;
}
declare module 'test-module' {
    export import Foo = require('test-module/Foo');
}

Works:

declare module 'test-module' {
    class Foo {
        public name: string;
    }
    export = Foo;
}

[csnover]

@mtraynham Yes, primarily the difference is that dts-bundle doesn’t use the language services; instead, it does a bunch of regexp stuff, which is bad. I would have just submitted dts-generator as a replacement to that project but there are a variety of reasons why that wasn’t possible.

However I do want to note that the code you wrote there isn’t the same; the “works” code couldn’t possibly work with that “driver” code unless WebStorm is terminally broken as it’s exporting Foo as the value of 'test-module', not as a property of the module.

[mtraynham]

@csnover awww, you know, you are right... My fault. So if you export = on your classes, and then export import at the top module... what's ultimately being exported? To your point, it doesn't seem like it's the class.

[csnover]

@mtraynham—

export import Foo = require('Foo'); is equivalent to this CJS code:

var Foo = require('Foo');
exports.Foo = Foo;

import Foo = require('Foo'); export = Foo; is equivalent to this CJS code:

var Foo = require('Foo');
module.exports = Foo;

[mtraynham]

Thanks, that clears up a few things... I think I was getting confused with the grunt-ts export transformer, which generates exports for you in the form:

import Foo_file = require('Foo');
export var Foo = Foo_file;

The .d.ts generation for this (generated by TypeScript) then looks like:

import Foo_file = require('Foo');
export declare var Foo: typeof Foo_file;

I think this was a workaround for #512, but it has some weird consequences down the line when you use it with dts-bundle or your own dts-generator... atleast in Webstorm.

[csnover]

In bb307f81639b2bc29a86aef5f92492e38e88ba67 the functionality that dts-generator used to get access to emit files was removed. What is the intended replacement functionality? Just program.emit(sourceFile, writeFunction)?

[mhegazy]

@csnover yes, we moved emit to program.emit. that is not going to help you though. we should expose emitFile, getEmitResolver, and possibly getEmitHostFromProgram in the .d.ts. that should be fairlly simple, just a change to the jake file. i have logged #2217 to track it.

[csnover]

@mhegazy I seem to be able to do what I need to do with just program.emit and don’t need those others. https://github.com/SitePen/dts-generator/commit/0be96b6f064cfb88558ba93d18d42f9e235c232f

I do need #2139 to land sooner than later though since declaration emits are broken right now with ES6 imports. :)

[ToddThomson]

https://github.com/ToddThomson/tsproject/commit/4979240232f0237e1a321e90a1e3c22debaa650e

Still testing, but the build directory in the provided sample shows single file declaration.

[niemyjski]

I was able to get bundling working on my project! It use ES6 Modules and bundles using TSProject!!!

[lazdmx]

:+1:

[comdiv]

My suggestion is that this scenario is available if you build your solution in single file: 1) Pure typescript modules as source (they allow simple merging) 2) Using --out MODULENAME.js --module AMD For now it generates something like this: var MODULENAME; (function (MODULENAME) { //BODY OF FILE 1 })(newthe || (newthe = {})); var MODULENAME; (function (MODULENAME) { //BODY OF FILE 2 })(newthe || (newthe = {}));

So as u see - we got valid single JS module that can be used with AMD with shim:{MODULENAME:{exports:"MODULENAME"}}

My suggestion to allow TS generate fully complicated AMD/UMD modules:

file A:
///<reference path="lib1.d.ts"/>
module X {
  export class A{}
}
file B:
///<reference path="lib2.d.ts"/>
module X {
   export class B{}
}
file C:
///<amd-dependency path="lib1"/>
///<amd-dependency path="lib2"/>
export module X;

1 Compiler should fail if TWO export module with same name exist
2 Compiler should add export module definition in tail of --out FILE :

define("X",["require","exports","lib1","lib2"],function(require,exports,lib1,lib2){
      exports.X = X;
});

3. Compiler should hide var X from global scope, result file must be rewrited :
(function(){
var X;
(function (X) {
   //BODY OF FILE A
})(X|| (X= {}));
var X;
(function (X) {
   //BODY OF FILE B
})(X|| (X= {}));
define("X",["require","exports","lib1","lib2"],function(require,exports,lib1,lib2){
      exports.X = X;
});
})();

What is the result:
1) Sources of module are TypeScript-style - no order, no dependency - simple merging
2) Export AMD module as result of compilation
3) No artifacts in global scope

It can be cause some issues to provide "lib1","lib2" instances to internal module, but i think it can be solved with "export initializer" something like that:
///<amd-dependency path="lib1"/>
///<amd-dependency path="lib2"/>
export module X(lib1, lib2) {
      A.setServices(lib1,lib2);
}
So in result it must generate something like that:
define("X",["require","exports","lib1","lib2"],function(require,exports,lib1,lib2){
       X.A.setServices(lib1,lib2);
       exports.X = X;
});

So we can provide any module-level initialization module on real phisical module binding.

For now i do it manually and then minimize.

[AsherBarak]

Why not consider tooling? Group .ts files into modules using directory structure or comments or whatever code-indifferent mechanism, and have the tool rewrite the imports and the exports into a new .ts file to keep everything going. In case of conflicts, emit errors. Tooling should also emit source maps. This eliminates the need for braking changes in the code and allows room for errors in the merging process. As all current and proposed modules loading relies on files location, and loading files is something you might want to manage to balance files sized with number of calls, this will keep with same principal while separating the development directory structure from the delivered modules structure.

[kripod]

Proposal suggestion

(Based on #5085, inspired by the @import function of SASS)

I would like to propose a mechanism for creating partial files, which are not compiled by default, but embedded into files which reference them. It's just like bundling, but with less hassle and in a more unified approach.

As a brief example, suppose that we have 4 files:

• index.ts
• user.ts
• _auth.ts
• _position.ts

The index.ts and user.ts files can reference contents of a partial file by using simple imports. For example, if we want index.ts to reference _auth.ts and _position.ts, use the following code in the non-partial index.ts file:

import auth = require('auth');
import position = require('position');

The content of partial files should be copied and instantly evaluated in their container file. For instance, whether we would also like to use the _position.ts partial file in user.ts, use an import just like above:

import position = require('position');

Although this may seem like a regular import, it's basically an include function which copies the partial file's full content at the place desired. That means, partial files should not be reused in multiple containers, but should be used for code organisation purposes. When both index.ts and user.ts are referenced in a HTML file, the content of _position.ts will be duplicated and inserted into the 2 containers separately.

Last but not least, in order to keep backwards compatibility, there should be a tsconfig setting (compilerOptions.ignorePartialFiles) which could be set to true in order to keep compiling files which start with a _.

[ToddThomson]

@kripod https://github.com/toddthomson/tsproject already does this. It adds bundling to the tsconfig.json file and will build a bundle from modules using ES6 syntax. TsProject is a gulp adapter that can be installed with npm.

[kripod]

@ToddThomson I have seen that project before proposing my specification, but bundling from a single file seems to be overcomplicated for such a little effect.

[ToddThomson]

@kripod No that is not how it works. Your Typescript ES6 exported modules are defined in separate files. When you specify a bundle it works out all the dependencies and build a single bundled ts, js and d.ts. Take a look at the sample on the github tsproject repository.

[mironx]

I am in the same situation as @park9140 and I am little tried. I trayed the same solution (and also others). I have more experience in C# and Java and probably my notation about how it could works has influence of it.

But I dream about something similar to .net assembly which could have multiple files and I could load it.

Thant means would be good have that java script external module be like assembly. Even I our external module is big we could use in it namespaces [Namespace in module]. All files "are compiled" to one file. We could tell compilator: ok: it is my new super external module and it consists these 350 files. [Tell that this files build module] And these elements please make available see by other modules [Export Items] Between files should be possibility to have references [Reference inside module].
And supporting compilation exception when e.g. in one module would define two classes with the same name [Supporting Compilation Exception].

That meas we could indemnify 4 issues: Namespace in module Tell that this files build module Export Items Reference inside module *Supporting Compilation Exception

Generally I would like to focus on developing business logic, provide solution than focus on compilation process.

[mhegazy]

@mironx have you looked at browserify or webpack?

[mironx]

@mhegazy Thank your encouragement. I had know this tool and I had afraid bit of configuration - that it could be to complicated.

Thanks your encouragement I the ice has broken up :-) and I have spend several weeks with webpack and also systemjs.

What I would like to achieve?

But first what I would describe what I need to achieve.

I would like to create my suite of component which would exists in java script file. E.g.:

• lib1.js, lib.d.ts
• lib2.js, lib.d.ts
• lib3.js, lib.d.ts this library could be depend on angular, reactjs, kendo and others.

apps and apps which could consume this libraries and could have own libraries.

• app1.js libapp1.js libapp2.js and dependencies: lib1.js with lib1.d.ts, angular etc....
• app2.js ... silimar like app1

It would be like: software suite, framework or components.

1. One library could contain many external modules.
2. app it is just main js file

My adventure with webpack

I have tested webpack with grunt. I create GruntFile.js which could be use by each project. This GruntFile.js use project.json which indicate how to process structure of project (solution line in Visual Studio)

Problems

• a) I could join js files (after compilation of typescript) but I have problem with generated one type script date type file d.ts for library As I wrote I would like to have references between libraries - I need also d.ts
• b) I have problem to dynamic loading js to achieve the same as in systemjs. That mean to have one file in html.
• c) And with references. because I had to use external - I don't wont bundle vendors and library to one file.

Summarization

It this moment the biggest problem for me is point "a".

Main requriment I could compile whole libraries for each app but I need that my library work as plugin. Thant means for existing application someone else could provide library as plugin which extend functionality. This library would implement app interfaces to extend functionality.

New version of typescript compilator

I looked on roadmap of typescript compilator and I noticed interesting issues:

• 2.1 Support for project references
• 2.0 Improve lib.d.ts modularity
• 1.8 Concatenate module output with --outFile (only for amd and commonjs)

Conclusion

• Maybe in this moment it is difficult to achieve this structure which I was described.
• Maybe I am on wrong path because I have habit e.g form .NET or java and I would like transfer "assembly" to java script world But maybe no - typescript roadmap looks like small steps in this direction.

Maybe crazy idea for typescript compilator

What do you think to put to to output java script file declaration? As metadata in comments. It would transparent for java script. But js could be add as reference to typescript project - similar to assembly or jar file which has own metadata.

Architecture for enterprise, long term solution

In new year I and my team I will have make some architect decision - and I ma responsible of consequences.

I really appreciate you for each comments and criticism in this topic. Typescript architecture for applications and libraries. Thant means how to organize compilation process to achieve these goals.

I think thant others have similar problems with typescript projects - how to organize it when structure is more complicate, e.g.: #909 , #557

Thank you and sorry that I extended this issue of details which could be out of scope

[mhegazy]

@mironx, would not https://github.com/Microsoft/TypeScript/pull/5090 be sufficient for your needs?

[mhegazy]

ES6 modules provide the ability to build modules from smaller ones using export * from "mod" and export { a as b } from "mod". This allows for having the multiple modules for implementation purposes, but then exposing a single "entry point" module that collates all of the smaller ones into a complete unit. This would be the recommended route for this scenario.

A follow up feature for the TS compiler is https://github.com/Microsoft/TypeScript/issues/4433, to enable publishing a single .d.ts file that has the "shape" of the "entry point" module, and thus completely hide the internal modules from consumers.

[Clark159]

This is my solution uses TypeScript 1.8, The solution build multi-Class in a module.

• sample : https://github.com/Clark159/clark159.github.io/raw/master/_posts/%5BTypeScript%5D/%5BTypeScript%5D%20%E5%BB%BA%E7%AB%8B%E8%88%87%E4%BD%BF%E7%94%A8AMD%20Library/Lab.zip
• document : https://dotblogs.com.tw/clark/2016/04/25/124833