Support server packages

[guybedford]

Make these use the node core libs when run on the server - https://github.com/jspm/npm/blob/master/npm.js#L19

[guybedford]

This has been added now with nodelibs@0.0.6, SystemJS master and jspm-npm master.

This can be tested by linking jspm-npm master into jspm, and installing SystemJS to master (npm install systemjs/systemjs#master).

Then the following will work:

import childProcess from 'nodelibs/child_process'

As well as other server requires.

Let me know how it works out.

[OliverJAsh]

I followed your instructions, but got this error:

❯ ./node_modules/.bin/jspm install nodelibs

     Updating registry cache...

warn GitHub credentials not provided so rate limits will apply.
     Use jspm endpoint config github to set this up.

     Looking up github:jspm/nodelibs
     Looking up npm:Base64
     Looking up npm:base64-js
     Looking up npm:ieee754
     Looking up npm:inherits
     Looking up npm:pbkdf2-compat
     Looking up npm:ripemd160
     Looking up npm:sha.js

err  Package.json meta not provided in endpoint request

warn Installation changes not saved

[guybedford]

Perhaps try reinstalling. It seems like it may be using an outdated version of the GitHub endpoint perhaps?

[guybedford]

Actually this could be the npm endpoint.

[guybedford]

It would help to confirm the versions of jspm, jspm-npm, jspm-github if possible.

[OliverJAsh]

jspm@0.8.5 jspm-npm@0.11.1 (git+https://github.com/jspm/npm#14562fff3353e1b6cf2634a9a7e8f1930e91654e) (dependency of jspm; master) jspm-github@0.8.1

I can try re-installing everything.

[OliverJAsh]

Same problem when re-installing.

Here were my steps:

npm i jspm --save
cd node_modules/jspm
npm i jspm/npm#master --save
cd ../..
./node_modules/.bin/jspm init
./node_modules/.bin/jspm dl-loader -e
./node_modules/.bin/jspm install nodelibs

[guybedford]

Ok this seems like it is because jspm-npm should be at version 0.10.0 against jspm 0.8.5.

[guybedford]

Ahh right - this needs to be tested against jspm@beta.

Be sure to check the beta release notes - https://github.com/jspm/jspm-cli/releases/tag/0.9.0-beta.2.

[OliverJAsh]

Ahh, so I actually need:

jspm@0.9.0 (beta) jspm-npm@0.10.0

Is that right?

[guybedford]

Yes - it may be possible to back-merge the npm changes to 0.10.

[guybedford]

Argh, sorry not reading thoroughly - you actually want jspm@0.9.0 beta against the jspm-npm@0.11.0 it's just 0.8 that needs the lower jspm-npm which was the issue.

[OliverJAsh]

Got it, thanks. Now running:

jspm@0.9.0-beta.2 jspm-npm@0.11.0

import fs from 'nodelibs/fs';

console.log(fs);
// => { readFileSync: [Function] }

That's working, albeit only with a partial implementation.

However, if I try child_process:

import childProcess from 'nodelibs/child_process';

console.log(childProcess);

I get:

ReferenceError: Error loading "github:jspm/nodelibs@0.0.6/child_process" at file:/Users/Oliver/Development/tmp/jspm-server/jspm_packages/github/jspm/nodelibs@0.0.6/child_process.js
Error loading "github:jspm/nodelibs@0.0.6/child_process" from "app" at file:/Users/Oliver/Development/tmp/jspm-server/app.js
Error evaluating file:/Users/Oliver/Development/tmp/jspm-server/jspm_packages/github/jspm/nodelibs@0.0.6/child_process.js
nodeRequire is not defined
    at Object.<anonymous> (evalmachine.<anonymous>:3:18)
    at evalmachine.<anonymous>:5:4
    at doEval (/Users/Oliver/Development/tmp/jspm-server/node_modules/systemjs/dist/system.src.js:2212:10)
    at __eval (/Users/Oliver/Development/tmp/jspm-server/node_modules/systemjs/dist/system.src.js:2126:7)
    at Loader.exec [as __exec] (/Users/Oliver/Development/tmp/jspm-server/node_modules/systemjs/dist/system.src.js:211:5)
    at load.metadata.execute (/Users/Oliver/Development/tmp/jspm-server/node_modules/systemjs/dist/system.src.js:1077:16)
    at linkDynamicModule (/Users/Oliver/Development/tmp/jspm-server/node_modules/systemjs/dist/system.src.js:539:32)
    at link (/Users/Oliver/Development/tmp/jspm-server/node_modules/systemjs/dist/system.src.js:402:11)
    at Object.execute (/Users/Oliver/Development/tmp/jspm-server/node_modules/systemjs/dist/system.src.js:714:11)
    at doDynamicExecute (/Users/Oliver/Development/tmp/jspm-server/node_modules/systemjs/node_modules/es6-module-loader/dist/es6-module-loader-sans-promises.src.js:650:25)

[guybedford]

Did you run jspm dl-loader -e?

[guybedford]

Thanks - will work on improving the fs implementation.

[OliverJAsh]

Ahh, because I'm loading from the server, I'm downloading SystemJS from npm. I tried to load SystemJS from jspm_packages instead, but didn't get very far:

var System = require('./jspm_packages/system');

It fails to load ES6 Module Loader.

I just did npm i systemjs/systemjs#master --save and that fixed my error.

[OliverJAsh]

If you have any tips on how I can load SystemJS into Node from jspm, I'd love to hear them.

[guybedford]

Oh of course! Sorry for the headache have fixed up the instructions above.

jspm master has jspm run now - perhaps try that?

[OliverJAsh]

I'm running this on the server with Node. Should I be seeing the whole fs? (I thought we only mocked these modules if they couldn't be required from Node directly.)

[OliverJAsh]

jspm run works nicely. Thank you!

[guybedford]

Great to hear that. Unfortunately the conditional loading is a scenario dependent on the new conditional loading feature of the loader itself, which hasn't been released yet.

At the moment we have:

• Server libraries working within nodelibs
• Client libraries that also happen to work on the server from nodelibs

Where the client shims are not suitable for the server is the case we have to tackle.

In the mean time, perhaps use the nodeRequire directly when you need Node core libs (although not something we want to be doing at all) - var fs = nodeRequire('fs').

[OliverJAsh]

Is nodeRequire added as a global? (I can't see it)

[guybedford]

Ah, it will only be present for CommonJS, as the assumption was it would not be used by the user so was only needed in these cases.

You'll need to ensure the module is being interpreted as CommonJS by having a require statement or module.exports present.

[OliverJAsh]

I see. Thanks.

I've creating a demo for a server package with the necessary setup and run instructions at https://gist.github.com/OliverJAsh/ed55fbad3b715834e49b, for anyone who follows along.

I understand we're still pending conditional loading of real node lib vs. mock on browser. Is there anything else still pending? Are all the mocked node libs feature complete? I will have a go at moving a real server package to SystemJS if it's ready!

[guybedford]

Thanks so much for diving into this stuff! I'm not aware of how the browserify node libs behave on the server and how feature-complete they are, I've simply just pulled them through.

It is still really early on these use cases, so it will be a little painful - will do my best to keep up with you here. It helps a lot to understand the requirements, so thanks for reporting back on this stuff, completely understand if you want to wait a little before attempting further too.

[OliverJAsh]

Could you update me when we have conditional loading of real node lib vs. mock on browser? Is there an issue I can track for that?

[guybedford]

I've added this full support now in jspm-npm@0.12 and the master branch of jspm through jspm run. Code can be written like:

  jspm install nodelibs
  jspm run app

app.js

import fs from 'nodelibs/fs';
console.log(fs.readFileSync('./file.txt').toString());

etc.

This is a hack for now, but will work and allow code to run in the browser and server.

True handling of the conditionals will be cracked when we finally get conditional loading into the loader itself.