Windows .net build process

[avanderhoorn]

Just wondering if we can work to get windows .net compatible build process up and running that can work with r.js.

Specifically, I want to be able to run my build process using the standard .net dependencies.

A way of doing this might be to use one of the following Interpreters/Run Times to handle the interop between .net and executing the r.js script:

• http://jint.codeplex.com/
• http://javascriptdotnet.codeplex.com/
• http://jurassic.codeplex.com/

There maybe other ways of doing this, but the core concept is that a .net developer shouldn't have to install node/java just to get this build step running.

Cheers Anthony

[jrburke]

Thanks for starting this thread. Some notes:

It would be good to see if any code from #37 would be applicable here. I suspect not directly, but mentioning it because it was looking into other environments besides node and rhino.

For .net: if the user needs to install one of those items above, I see it as less utility than either not installing any dependencies or asking for a node install.

Is there something we can use that would not require the user to install something else?

For instance, maybe use Windows Script Host in some way to drive the JS script?

The main thing is getting a file IO API in that JS environment, so that r.js can do the file combination/writing out built files. So that is probably the main factor in choosing an approach. If a jscript file run through WSH has a way to do file IO that may be a way to start.

[junosuarez]

Chutzpah is a popular .Net utility for running JS tests in a .Net-centric build environment - we use it at Healthwise. It uses PhantomJS as a headless browser, which uses Webkit JavascriptCore as it's JS engine. That might be a good approach, and I have some ideas for some exploration I'll do this evening.

It would also be great to look into integrating with the new ASP.Net MVC 4 build / bundle filters to a achieve a real .Net native platform implementation that fits well in a .Net workflow.

[jrburke]

@jasondenizac ideally we would not have to require the developer to install anything -- in that case I'm fine just suggesting people install nodejs. Hopefully there would be enough installed tech we could get it to work without needing an install of a helper lib.

I'm saying this though being very new to .net development, so I may not have the culture correct.

[junosuarez]

Most .Net devs would be most comfortable having a single .exe CLI file with the same options as r.js.

To clarify, a .Net dev would be thinking in terms of the optimizer being its own self-contained tool, rather than a script with dependencies. I agree that the best developer experience doesn't require them to install other tools or dependencies that they wouldn't otherwise already have installed.

[jrburke]

@paulogaspar7 on twitter suggested these two links for doing file IO in windows script host: http://msdn.microsoft.com/en-us/library/d6dw7aeh%28v=vs.85%29.aspx

example usage: http://msdn.microsoft.com/en-us/library/czxefwt8%28v=vs.85%29.aspx

[johnpapa]

I'd love to see options so you don't have to use r.js and instead can use your choice of minification and bundling, such as ASP.NET MVC 4's, or squishit, cassette, etc. Not been able to do that cleanly yet myself.

[avanderhoorn]

I think we need to look at what problem r.js.

For me the major problem solver here is how do we manage the optimized build. How do we facilitate using r.js to walk our dependencies and create a single file. I guess one could say that this is all about custom "bundling", where most bundling solution just go by folder or what ever, whereas, r.js is walking dependencies.

If I have this correct, once we have this solved, post this step we can run what ever minification engine we want. We can also just wire up what ever solution we have into VS2012's support for custom bundling and allow .net devs to do this at run time or build time.

If all the above is correct, then we just need to look at what r.js needs to run. If it just needs a V8 engine to run against then http://javascriptdotnet.codeplex.com/ provides the support that we would need here. Then we just need to work out how to feed it the r.js file and then I guess make sure that it has the ability to read files from the file system.

Is there something I'm missing here or is this kinda how it lays out?

[jrburke]

@avanderhoorn that is my understanding of the goal. Other minification/bundling tools are usually not aware of AMD modules and how to trace their dependencies, and how to name anonymous modules when they are bundled together.

For any non-node/rhino environment, we basically need to create some environment shims similar to the modules here that are used for Node:

https://github.com/jrburke/r.js/tree/master/build/jslib/node

Currently I'm favoring windows script host since it will not require another install of something (hopefully).

[wardbell]

I do not understand why the order of AMD scripts in a bundle should matter. Compliant scripts should execute one and only one method: the define method that identifies dependencies and supplies the function to call when THIS component is required.

Shimmed, non-compliant scripts are a different matter; I might expect them to be problematic and they might have to be bundled in a particular order (I'm thinking in particular of plug-ins to jQuery, KO, etc.).

[junosuarez]

@wardbell r.js is doing more than concatenating files together - it's parsing the AMD syntax, walking the dependency graph, adding those modules, adding the correct name to the define call, call the write function of any loader plugins to optimize other dependencies such as templates, etc.

It would be great to take full advantage of r.js and just find a way to wrap it in a .Net-environment-friendly format.

[JamesMGreene]

Lest not we forget, of course, that not all .NET developers are running Windows either. Not saying we need to address that audience so much as that the requirement wording is slightly inaccurate. :)

[jrburke]

@JamesMGreene I adjusted the title and summary to reflect the windows focus of the ticket.

[junosuarez]

I spent a little bit of time experimenting and came up with this: https://github.com/jasondenizac/RN

I noticed that the concept of environment is fairly pervasive in r.js, and there is some opportunity for further refactoring to make it less lightly coupled to either node or rhino. For now, the approach I took for the proof-of-concept was to masquerade as node and write the appropriate stubs or facades. I have more work to do on some of the file access methods to run the optimizer, but in its current state it can run an AMD program by rn main.js, equivalent to node r main.js.

[EisenbergEffect]

@jasondenizac That's fast work. I looked at your implementation and it looks pretty straight forward. I'll try it out on a real app in the next week and get you some feedback (or pull requests).

[avanderhoorn]

@jasondenizac Not that I've run it yet, but shimming out node the way you have looks like a great way of solving the problem and bridging the gap between node/js and .net.

@jrburke I know you have put a next tag on this one, but I'm just wondering when you think we will move to formalize some of this?

[junosuarez]

@avanderhoorn That project doesn't actually work yet, it was just a proof of concept. Ultimately, I think trying to shim node is the wrong approach. There is a lot of wrapper code in r.js that tries to detect the environment between node and rhino - it would be nicer if it expressed a dependency on an interface for the file system, etc, functionality that it needed so that it could be more easily adapted to different environments (and integrated more easily into other build systems, especially IDEs, which was my ultimate goal for the .Net stuff)

[EisenbergEffect]

I agree. I messed with the code over the last couple of days and determined pretty quick that r.js was making some serious assumptions about it's environment (which caused it to require modules which weren't present.) I'd love to see a formal interface between r.js and it environment so we can implement that in different ways.

Also, I have a bunch of ideas related to how this might be consumed by .net devs and I'm happy to contribute on that end.

[avanderhoorn]

Ok cool, so it seems like the next step is for r.js to have these environment assumptions abstracted out.

@jrburke is this something that you are onboard with and agree with in terms of approach? If so when are you thinking something like this might be able to happen?

[junosuarez]

The internals actually already have these abstractions, but the entry point into r.js is doing environment detection stuff from the very first IIFE. Ideally, r.js should still be self-contained for the most common use cases (node and rhino, I suppose) but have some sort of detection for the interfaces to be provided by the environment - similar to define.amd detection.

[jrburke]

My plan for this is to create a windows script host detection bit in the first IIFE as @jasondenizac mentions, which would then set it up the "env!" plugin references would load the environment-appropriate modules for that environment. So, in this directory:

https://github.com/jrburke/r.js/tree/master/build/jslib

there would probably be a "wsh" directory that would contain the adapter modules, similar to the set that are in the node and rhino directories in the above directory, but that use the capabilities wsh has access to. In particular, the wsh/file.js would hopefully use the FileObject as mentioned here:

http://msdn.microsoft.com/en-us/library/d6dw7aeh%28v=vs.85%29.aspx

As for timeline, the "next" label I'm using to mean "this would be great to get in the next release if there is room, but it should not block". At this point, I will probably do the a 2.0.3 release without this change as the window between 2.0.2 and now is getting longer and more people are stumbling over some of the basic bugs logged for 2.0.3.

It is not likely that I will get to this in July. I have some work stuff to do, a talk, and some family visiting, which will take most of this month.

I'm happy to give some pointers to anyone who wants to try the approach I just listed out, just be aware of some notes on the contributing page about the CLA stuff.

Some pointers:

https://github.com/jrburke/r.js/blob/master/dist.js

That file builds r.js from its components. Notice the envs array.

https://github.com/jrburke/r.js/blob/master/build/jslib/x.js#L35

is the beginning of the environment detection, and

https://github.com/jrburke/r.js/blob/master/build/jslib/x.js#L109

Has some special INSERT comments, one for each environment branch. There should probably be something there for the wsh environment.

In https://github.com/jrburke/r.js/tree/master/build/jslib there should be a wsh.js file that provides similar implementations of the functions as done in the sibling node.js file.

The https://github.com/jrburke/r.js/tree/master/build/jslib/node directory should be copied to a wsh directory and then each of the modules in there adjusted so that their implementation does something appropriate for the wsh environment.

https://github.com/jrburke/r.js/blob/master/build/jslib/env.js needs to have a wsh environment detect added to it.

That is my first thought on it though, open to other ideas.

[junosuarez]

@jrburke In dist.js, does the order of includes in the libFiles array matter?

[jrburke]

@jasondenizac it does a bit, well at least with requirejs 1.0 it did, it may matter less now with 2.0.

[jdanyow]

hey guys, is this still in progress?

[junosuarez]

@jdanyow alas, I no longer have a windows machine to try this on. i think a good way to pursue it would be to continue abstracting out the stuff that's node/rhino specific and add support for WSH, rather than trying to emulate node. The TypeScript compiler does a good job of abstracting out compiler dependencies to run on WSH or node. Unfortunately, codeplex doesn't have a way to link to specific lines, but see http://typescript.codeplex.com/SourceControl/changeset/view/2bee84410e02#bin/tsc.js and search for var IO =

[ProVega]

Any updates on this? I want to echo jden here and say that approach TypeScript uses works VERY well for MSBUILD/VS developers. We got TypeScript integrated into both Visual Studio and the Builder server with very little effort.

[EisenbergEffect]

I'd love to have r.js running on windows without node...and I'd be willing to put some money towards seeing that happen. I might know some other companies that would as well.

On Thu, Jan 24, 2013 at 12:06 AM, ProVega notifications@github.com wrote:

Any updates on this? I want to echo jden here and say that approach TypeScript uses works VERY well for MSBUILD/VS developers. We got TypeScript integrated into both Visual Studio and the Builder server with very little effort.

— Reply to this email directly or view it on GitHubhttps://github.com/jrburke/r.js/issues/201#issuecomment-12637559.

Rob Eisenberg, Blue Spire Consulting, Inc. Caliburn Project 850.264.3996

[jrburke]

It is not so much a money matter, but a time/support issue and knowing the environment enough to get it to work. However @jden may have pointed to a link that shows the ActiveX objects to use, so putting this in the 2.1.5 bucket for consideration. It may get bumped to a future release though depending on other 2.1.5 factors.

I am also willing to give feedback to anyone who wants to do a pass at a pull request.

[EisenbergEffect]

It's a very valuable feature, that's all. So, if money can make it happen sooner than later, it's available ;)

On Thu, Jan 24, 2013 at 1:26 PM, James Burke notifications@github.comwrote:

It is not so much a money matter, but a time/support issue and knowing the environment enough to get it to work. However @jdenhttps://github.com/jdenmay have pointed to a link that shows the ActiveX objects to use, so putting this in the 2.1.5 bucket for consideration. It may get bumped to a future release though depending on other 2.1.5 factors.

I am also willing to give feedback to anyone who wants to do a pass at a pull request.

— Reply to this email directly or view it on GitHubhttps://github.com/jrburke/r.js/issues/201#issuecomment-12665476.

Rob Eisenberg, Blue Spire Consulting, Inc. Caliburn Project 850.264.3996

[jrburke]

I have started a branch for this, but it is not done: https://github.com/jrburke/r.js/tree/wsh

r.js runs, but there are errors during runtime. I am having trouble figuring out how to debug, so any help on how to do that would be appreciated. What I am trying to do:

cscript //d //x path/to/r.js -o test.build.js

However the //d option for cscript does not trigger a debugger. I have "Visual Studio Express 2012 for Web" installed and running, but no project, and I cannot seem to create a simple project that just calls out to cscript, all the templates are for "Visual Basic" and "Visual C#".

[billti]

I believe CScript can be picky about forward/backslash direction. The above switches look right, but try with path\to\r.js.

Also, TypeScript runs on Node and CScript and abstracts away the IO. See http://typescript.codeplex.com/SourceControl/BrowseLatest#src/compiler/io.ts .

[jrburke]

@billti the cscript command runs as I have it listed above or as path\to\r.js, same effect: no jumping in to the debugger.

I have the IO adapters for cscript in place, just having trouble debugging their use within the r.js code. I am getting odd test errors and the errors do not have any stacks to show precise locations (and what called those locations), so was hoping to jump into a debugger to figure it out.

I just cannot seem get a debugger attached to that cscript command. Possible issues I have thought of:

• Maybe the "Microsoft Visual Studio Express 2012 for Web" does not actually ship with a JS debugger
• Maybe that debugger is not wired up in the registry in a way that cscript knows that it is available
• Maybe cscript just does not pay attention to those switches any more in Windows 7

I am not sure though, just guessing. It is hard googling for these things too since either it is vbscript results or the "js debugging" ones talk about the IE developer tools.

[billti]

Ah, right. I'm not sure the Express editions will work for that. You might need Pro edition (or better). I don't suppose you have a copy knocking around (or a trial edition installed)

Else if you can give me the repro steps I can try to figure it out.

[billti]

I just did a quick poke running with no args, and running with "-o ". A couple of exceptions that were occurring that probably wouldn't hit in other environments.

"0x800a01b6 - Microsoft JScript runtime error: Object doesn't support this property or method."

This occurred on Object.defineProperty (line 13641). CScript doesn't support ES5, so these methods don't exist on Object. This occurs in the Mozilla source-map module (the block to "define('source-map/source-map-consumer'...").

I also saw a couple of "0x800a0009 - Microsoft JScript runtime error: Subscript out of range". This is mostly from code such as "WScript.Arguments.Item(n)", where if n >= WScript.Arguments.length it throws (rather than the common JavaScript behavior of returning undefined if indexing past the end of an array).

I'm guessing you may be well beyond these initial issues however, so as mentioned above, if you want to tell me how to hit the issue you are stuck on, I can try and tell you what's crashing.

[jrburke]

@billti thank you so much for trying it. I mostly want to figure out how I can debug it as I expect over time I will need to do fixes and want the right tools for it.

Can you confirm what Visual Studio version worked for you to get into the debugger? Also which version of Windows (I'm using 7 but can switch to 8).

[billti]

I run VS2012 Ultimate on Win8 in general on my machines. I'm pretty sure Win7 or Win8, and VS2012 or VS2010 will work, provided you're not using the free 'Express' editions of VS (i.e. Pro, Premium, or Ultimate should all be fine for debugging CScript via the //X switch).

[jrburke]

@billti thank you very much! Installing VS Pro did the trick. Would be nice if the express web version included a JS debugger, but such is life.

[jrburke]

I think I have hit a problem that makes it difficult to continue. cscript apparently has a small stack space. I am regularly getting "out of stack space" errors when doing a build. If someone else wants to try it, here is the current wsh branch snapshot.

To make further progress on this, I think I would need help with one of the following:

• A way to increase the stack space for cscript when calling it (something like the -Xmx arg when using Java)
• A way to call some other JS engine that is built into Windows, hopefully something that can be run from the command line, and does not require an install of software (otherwise, I would just suggest installing node and using r.js that way).

[liddellj]

For what it's worth - I've just started playing around with r.js on Windows, and was able to execute it simply by dropping node.exe alongside r.js and executing from the command line. I didn't have to install anything.

I think that's a perfectly acceptance experience on Windows.

[junosuarez]

@liddellj certainly node.js, in my opinion, is the easiest cross-platform way to run r.js. However, I say that now with the luxury of having been working at a node.js shop for almost the past year. I remember the last company I was at, and the overall team environment there would have been very hostile to having node.js as a dependency in our build process. There's still a significant developer population for whom having r.js available on Windows without node.js is a big plus. Perhaps someone in that type of position to comment further and suggest an acceptable path forward.

[AndrewSwerlick]

To add the comments @jden made, the newest version of asp.net has the capability to do some need on the fly minification and bundling of various css and js assets. The framework has some extensiblity points that let you write your own bundlers, so I'd love to create an on the fly require.js bundler, but it relies on developing some sort of C# only bundling process. Which is way I've been watching this issue closely, and trying to see how I might be able to help when I have time.

[junosuarez]

@AndrewSwerlick interesting use case - using r.js in-process from .Net. This would not be handled by having r.js run on wsh or cscript, to my knowledge. There's been a project called owin developed recently to run node.js and .Net code in the same process. From the readme it looks like the bindings are primarily in the other direction, to run .Net code from node.js, but it might be worth looking into.

It's similar to the approach I hacked on about a year ago which involved running v8 and calling it form within .Net - the results of that (incomplete) effort are at https://github.com/jden/RN

[jrburke]

My goal for this ticket is to not have to have people install something to get windows-based builds. However, I am open to saying that something like a .net version needs to be there. If someone knows of a way from the command line where I could invoke, for example, the IE JS engine from the command line, and it had access to the ActiveX object(s) for file IO, I am open to hearing about that. Something like:

\Some\AlreadyInstalled\.netWrapper \Some\JS\Component path\to\r.js -o build.js

[JamesMGreene]

Might be of interest: http://stackoverflow.com/questions/7167690/what-is-the-progid-or-clsid-for-ie9s-javascript-engine-code-named-chakra/7168837#7168837

This talks about executing scripts with IE9's Chakra engine from .NET (and, theoretically, from CScript). I would assume this has a prerequisite of having IE9 installed on the machine, though.

[AndrewSwerlick]

This looks promising, but I haven't tried it myself http://stackoverflow.com/questions/7167690/what-is-the-progid-or-clsid-for-ie9s-javascript-engine-code-named-chakra. Also this may be an easier way of interacting with the same thing https://github.com/Taritsyn/MsieJavaScriptEngine

[jrburke]

Those links look promising, but I think the lack of a progid may make it hard to use in practice.

I wonder if it is possible to use the IActiveScript stuff directly from a cscript script. Something where I could do new ActiveXObject() on something to get an IActiveScript, and somehow select chakra from that thing, then feed it a script to run.

[junosuarez]

@glennblock do you know someone who can lend a hand with some WSH expertise?

[rjgotten]

Using WSH for minification is not a good idea. As it is dependent on the currently installed version of IE (and Office or other Microsoft products for that matter) it can break unexpectedly and spectacularly on lack of ES5 support.

There may also be issues with build-server support; where the build agent needs the correct set of permissions to call out to either WSH or Chakra. (Btw, does hardcoding a COM GUID to call out to a fixed version of the Chakra engine leave anyone else with a sense of impending doom? Is that thing a well-defined, guaranteed to remain, constant? ...)

A portable Node.js is imho a far better solution. The entire MSBuild chain is (eventually) still based on commandline tools. It's not some magic black box; it is (relatively) easy to create a custom MSBuild target that calls out to portable Node.js with r.js.

I think I'm going to put in a request with my employer to contribute the build target I recently authored for internal use by the company. Maybe this will prove a useful solution to the ASP.NET+RequireJS community.

@AndrewSwerlick

the newest version of asp.net has the capability to do some need on the fly minification and bundling of various css and js assets. The framework has some extensiblity points that let you write your own bundlers, so I'd love to create an on the fly require.js bundler

Fair note of warning.. I had a quick look at the internals of the bundling feature using ILSpy to see if I could easily extend it to have a static version token in the URLs (instead of the forced hash/fingerprint in the query string) and the whole thing is a mess: it looks like it was cobbled together by an intern during summer.

[david-driscoll]

System.Web.Optimization is a mess, absolutely!

I know this thread is kinda old, but looking at something like JavaScript Engine Switcher, would this then give us the ability to add an appropriate shim to r.js?

I might be interested in trying get it working, and potentially create a pull request.

This could also work hand in hand with Bundle Transformer and then we could create a custom bundle for r.js, and then leverage other ASP.net features like VirtualPathProviders (for embedded script files, etc).

[jrburke]

I prefer to be able to use something that is installed by default by the OS, or perhaps something that comes by default with Visual Studio if that is used by most people. Otherwise, if something needs to be installed by the user before using r.js, I prefer to just suggest installing node.

[rjgotten]

I prefer to be able to use something that is installed by default by the OS, or perhaps something that comes by default with Visual Studio if that is used by most people. Otherwise, if something needs to be installed by the user before using r.js, I prefer to just suggest installing node.

That's the thing; You can install a portable distribution of Node.js as part of a NuGet package. The same NuGet package can contain a *.targets MSBuild file that can be woven into an ASP.NET project to perform pre-publish/pre-build/post-build (wherever you want it to go) optimization through the r.js optimizer. (Heck, you can also go the extra mile to weave the log output from Node.js back into the Visual Studio or TFSBuild logs, if you want. I did...)

In this case you can have your cake and eat it: one-click, fire-and-forget seamless integration.