WebPack support - Githubissues

IngwiePhoenix commented 9 years ago

I admit it. PHP still is a language I prefer over JS... And that is why I want to use it properly.

A while ago, I had discovered WebPack. It allows me to use Loaders in order to transpile stuff to pure JS. So I wanted to use uniter. Problem is, that even a simple bit of testing fails:

Ingwie@Ingwies-Macbook-Pro.local ~/W/BIRD3 $ uniter -r 'echo 7 + 2;' --dump-ast
fs.js:792
  return binding.lstat(pathModule._makeLong(path));
                 ^
Error: ENOENT, no such file or directory '/Users/Ingwie/Work/BIRD3/uniter.js'
    at Error (native)
    at Object.fs.lstatSync (fs.js:792:18)
    at Object.realpathSync (fs.js:1383:21)
    at modular.configure.transport (/usr/local/lib/node_modules/uniter/node_modules/modular-amd/index.js:57:31)
    at Module.util.extend.load (eval at <anonymous> (/usr/local/lib/node_modules/uniter/node_modules/modular-amd/index.js:23:25), <anonymous>:363:56)
    at Module.eval (eval at <anonymous> (/usr/local/lib/node_modules/uniter/node_modules/modular-amd/index.js:23:25), <anonymous>:277:40)
    at Object.util.each (eval at <anonymous> (/usr/local/lib/node_modules/uniter/node_modules/modular-amd/index.js:23:25), <anonymous>:44:38)
    at loadDependencies (eval at <anonymous> (/usr/local/lib/node_modules/uniter/node_modules/modular-amd/index.js:23:25), <anonymous>:276:30)
    at eval (eval at <anonymous> (/usr/local/lib/node_modules/uniter/node_modules/modular-amd/index.js:23:25), <anonymous>:340:29)
    at Funnel.util.extend.done (eval at <anonymous> (/usr/local/lib/node_modules/uniter/node_modules/modular-amd/index.js:23:25), <anonymous>:122:25)
Ingwie@Ingwies-Macbook-Pro.local ~/W/BIRD3 $ node --version
v0.12.2

Why isn't uniter working in NodeJS?

asmblah commented 9 years ago

Hi @IngwiePhoenix,

I've just tagged and published v0.2.2, which should hopefully fix the issue you were having when running Uniter on the command line.

I've added support for Browserify, but I've not actually tried WebPack just yet, please let me know how you get on and if you get stuck I'll take a look for you.

Cheers! Dan

IngwiePhoenix commented 9 years ago

Cool, now it works! :)

Now, in order to use WebPack with uniter, I'd need to know a few things:

Is there a way to compile a PHP script/string into JS using uniter? For instance, there is the ejs-compiled plugin for WebPack, which runs an EJS template through a compiler which returns a template function. Is something similar possible with uniter?
If not, then: Which files are required for browser usage of uniter?
How to access variables and functions from the created PHP engine, once a script was run?
How are `include/require(_once)" statements handled?
Can one export functions and objects into global scope?

Kind regards, Ingwie

asmblah commented 9 years ago

Thanks, I guess what we really need is some actual documentation - I'll try to put some together at some point!

Is there a way to compile a PHP script/string into JS using uniter? For instance, there is the ejs-compiled plugin for WebPack, which runs an EJS template through a compiler which returns a template function. Is something similar possible with uniter?

Kind of: the code usually only gets transpiled at runtime; it does eventually generate the final JS which then gets eval()'d but there's not (yet) an easy API to get at that code, it's definitely something on the roadmap though.

If not, then: Which files are required for browser usage of uniter?

dist/uniter.js is a Browserified bundle of the core code; it has a UMD wrapper so you should be able to load it in with WebPack. The entrypoint (if you want to compile everything down with WebPack) is js/main.js. If you just want to try it out without a module loader, the UMD wrapper will just create a global with the name uniter (this is the way the interactive demo currently works at http://asmblah.github.io/uniter/demo/interactive.html .)

How to access variables and functions from the created PHP engine, once a script was run?

You can wait for the program to finish executing by listening to the promise returned by Engine.execute(), and then use the (currently very verbose) API to get at any global variables, eg.:

var phpEngine = uniter.createEngine('PHP');
phpEngine.execute('<?php $planet = "Earth";').done(function () {
    console.log(phpEngine.getEnvironment().getGlobalScope().getVariable('planet').getValue().getNative());
});

will print Earth to the console.

Getting at global functions is slightly trickier, you can still get at them via the private API, eg.:

var phpEngine = uniter.createEngine('PHP');
phpEngine.execute('<?php function getIt() {return 7;}').done(function () {
    console.log(phpEngine.getEnvironment().state.getGlobalNamespace().getFunction('getIt')().getNative());
});

will print 7 to the console. Obviously the API for these two needs to be improved :)

How are `include/require(_once)" statements handled?

There is now an asynchronous Promise-based API for this, for example (setTimeout with 500ms delay added just to demonstrate, synchronous is fine too):

var phpEngine = uniter.createEngine('PHP');

phpEngine.configure({
    include: function (path, promise) {
        setTimeout(function () {
            promise.resolve('<?php return "Hello from ' + path + '!";');
        }, 500);
    }
});

// Print anything written to stdout to the console
phpEngine.getStdout().on('data', function (data) {
    console.log(data);
});

phpEngine.execute('<?php print "Required: " . require("test.php") . "!";');

will print Required: Hello from test.php!! to the console.

Can one export functions and objects into global scope?

Yes, you can expose them, eg.:

var phpEngine = uniter.createEngine('PHP');

phpEngine.expose({
    planet: 'Earth'
}, 'stuff');

phpEngine.getStdout().on('data', function (data) {
    console.log(data);
});

phpEngine.execute('<?php print $stuff->planet;');

will print Earth to the console.

Hope this helps, Dan

IngwiePhoenix commented 9 years ago

Hey.

That sounds very, very neat! :)

Hope to see the compile-to-function feature, because that one would be really nice to have!

What I was looking for was to do something like this:

<?php

// Run a JS native:
/*
    engine.expose({
        console: console
    });

    Notice: no extra param.
*/
$console->log("foo bar");

// Export a function:
// engine.expose({ exports: {} });
$exports->greet = function() {
    echo "Hello, world!\n";
}
?>

How would I get the exporting into global namespace to work?

Otherwise I think I now know what I need to know. :) Very awesome, going to run tests on using uniter in WebPack now!

IngwiePhoenix commented 9 years ago

Oh I forgot. How to export global classes? Imagine I wanted to export the console object as a class, with it's methods as static methods.

IngwiePhoenix commented 9 years ago

Okay, here is what i got.

WebPack understands AMD definitions, much like Browserify seems to. However, there is a catch:

define([
    'languages/PHP/grammar',
    'languages/PHP/interpreter',
    'js/Language',
    'js/Uniter'
], function (

This won't resolve. When using a non-prefixed path ("js", "languages"), then WebPack will assume that to be a module name, like NodeJS' require().

I have to look for a more sutiable fix, but what would be possible is:

define([
    '../languages/PHP/grammar',
    '../languages/PHP/interpreter',
    './Language',
    './Uniter'
], function (

... Though, I am sure I can find a way to tweak WebPack to magically understand this definition too.

BTW, when I included the Browserify version, I had about 786 KB added to my build. That...is kinda huge. That is why I am trying to require Uniter from it's core, because I have a feeling that Browserify inserted stuff that does not need to be there.

asmblah commented 9 years ago

Thanks @IngwiePhoenix. I've had a quick play with Webpack and put together a small demo: https://github.com/uniter/uniter-webpack, use it as you wish :)

I've had to hard-code the path mappings you mentioned in webpack.config.js for now, which isn't ideal, I'll look to move these into the core code in a way that Webpack can pick up automatically.

Cheers Dan

IngwiePhoenix commented 9 years ago

If I happen to come across something, I’ll make sure to let you know about it! :)

asmblah commented 9 years ago

Just in case you need to start bundling up a PHP class structure, I've put together a small example of how to bundle up a simple PHP app structure (https://github.com/uniter/uniter-bundle) - the PHP files in /php/* are bundled along with /js/* and the Uniter core code to produce a single JS bundle file, using a simple autoloader to link the two together.

I've added support for both Browserify and Webpack bundling, you can use npm run build or npm run webpack respectively.

Hope this helps! Dan

IngwiePhoenix commented 9 years ago

Hey.

Has there been any progress on "pre-compiling" the PHP code to JavaScript in order to reduce the required amount of code to be included? I have just looked at the bundled app and such and really like how they are pretty easy to use! :)

Kind regards, Ingwie

asmblah commented 9 years ago

Hi @IngwiePhoenix,

Thanks - yes, it's getting there but there's still a way to go to make the embedding API as seamless as possible - a lot of the code in that repo (and https://github.com/uniter/uniter-jquery) could do with being factored out into some reusable NPM modules really.

I'm afraid there hasn't been much progress on the compiling step - at the moment I'm just embedding the PHP files 'raw' in the built bundle - which obviously isn't ideal as they aren't being minified and contain a lot of whitespace, etc. and get transpiled on every page load. The focus is really on adding all the missing PHP standard library functions and the remaining missing language constructs rather than optimizing for compiled size or speed right now, but it definitely needs looking at as you say.

Having a pre-compilation step would be a little tricky at the moment, as the runtime code is tied into the transpiler, and needs factoring out really. I'm planning on having a setting at some point for disabling the embedded transpiler (ie. to disable eval()) at some point too, to shrink the compiled bundles down further again.

Would you be able to achieve what you're after with the PHP code being embedded raw, as in these repos, or would the generated size make it completely unviable until we have a static compile step?

Cheers! Dan

asmblah commented 9 years ago

Hi @IngwiePhoenix,

Just to give you an update on part of this issue you raised, being able to statically compile PHP->JS for eg. WebPack. I've just tagged v1.3.1, for which I've split out the PHP->AST parser and AST->JS transpile steps into two NPM packages (phptoast and phptojs), which you should be able to use like this:

npm install --save phptoast phptojs phpruntime

var phpParser = require('phptoast').create(),
    phpToJS = require('phptojs');

console.log(phpToJS.transpile(phpParser.parse('<?php print "Hello!";')));

Note that the JS code this generates will contain references to another library I've split out, phpruntime (as require('phpruntime')), so WebPack (in your case) will need to be able to find that when compiling your bundle. The documentation's still a bit sparse, sorry about that.

The relevant code in Uniter (which is slightly different as it needs to transpile on-the-fly) is here: https://github.com/asmblah/uniter/blob/v1.3.1/js/Engine.js#L51

IngwiePhoenix commented 9 years ago

Hey!

Thanks for letting me know about that! Thats osme really good news to read about. I just came out of the hospital and feel really stomped out so reading some positive stuff is nice. But dont worry, it was just a regular eye check to see if my eyes are all good :)

I already know how to pull in a runtime - I am doing that with OJ’s already! So this should be really easy. Ill hopefuly get back into the swing of code and back to being around regularily so I can give you some feedback.

Now that you are splitting packages out and obviously doing a refactor in a way, you probably would be good in turning Uniter into a sort of meta-package that contains most of the API but actually uses the new „child projects“. That, however, is just a suggestion. Ill give you a poke on Gitter when I am up on my feet again (:

Kind regards, Ingwie.

IngwiePhoenix commented 9 years ago

Also I love how the first project unintentionally is named "php toast". :) nice little easter egg, even if it maybe wasnt intended to be.

asmblah commented 9 years ago

Hi @IngwiePhoenix,

Glad to hear you're alright! Yeah, there's not much code left in the main Uniter project now, it's mostly some logic to handle the original API and some integration tests now (which could do with cleaning up a bit.)

Oh and yep, it was intentional ;)

Cheers!

asmblah commented 6 years ago

Hi @IngwiePhoenix,

I'm just going through and closing some old tickets - I know you are/were working on a Webpack loader, but I don't think that's what this specific GH issue was about, so I'm going to close it 😄

Feel free to reopen it if I missed something.

Thanks!

asmblah / uniter

WebPack support #18