Plugin structure

[mathiasrw]

Lets look into if http://rollupjs.org/ is useful

[nickdeis]

I think the main goal here is to write ECMA6 modules of ./src.

Benefits

• Future proofing
• Commonjs modules through transpiling with babel.js
• Webpack, Rollup, and Browserify can be use to run the build
• This we all benefit Rhino, which can use require modules
  • Currently, alasql can not be loaded in Rhino unless optimization is turned off

I think we should offer a few dists: UMD and AMD.

[mathiasrw]

@nickdeis Great to see some inputs

ECMA6 modules of ./src is for sure the right way to go. Have a feeling it will need a lot of restructuring. Do you have any experience with rollupjs? Any good inputs on what to remember when converting a codebase to ECMA6?

As I recall it UDM is both AMD and CommonJS compatible - if I am mistaken we truely need an AMD version...

[nickdeis]

A lot of experience with webpack and some with rollup. Could you possibly assign this ticket to me? I spent a few hours last night working on this. The simplest way I found was to simply have a folders for each module. Something like:

• ./src/yy
• ./src/engines
• ./src/Table.js

For big classes like Select, I even created a separate folder below that and have a index.js. I'm sticking to prototypes (as opposed to ECMA6 classes) because they allow me import something like groupBy and stick it on.

Also ECMA6 classes are pretty useless (unable to have private variables!).

~~About the latter issue: Since the codebase uses require("fs") sometimes, it has some issues. I'm creating separate files for Cordova, Metetor, Node, and Browser~~

UPDATE: This is only an issue if I use

import * as fs from fs

as oppose to

var fs = require("fs");

Didn't even know that :smiley:

Have a feeling it will need a lot of restructuring

Yep, I want to see if I can get a simple statement working.

[mathiasrw]

Sounds great!

I spoke with Agershun about it a while agoe - so he made folders in ./src to go along - but never got further. Please feel free to use or rename them.

I have tried to assing you - but for some reason I cant select your name in the list...

Update:

Found some of the talks about modules / structuring it as plugins: https://github.com/agershun/alasql/issues/283 + https://github.com/agershun/alasql/issues/94

[nickdeis]

I can understand @agershun sentiments in these tickets, I've been looking around at tickets as well. Isn't there a 300k milestone marker that has a lot of tickets related to this idea?

[nickdeis]

@mathiasrw See https://github.com/nickdeis/alasql/tree/develop/ecma6 for the modules (work in progress) And for a small tech demo + test see https://github.com/nickdeis/alasql/tree/develop/ecma6/entry

[mathiasrw]

[nickdeis]

Found some other tickets along this line:

157 + #192

[nickdeis]

@agershun + @mathiasrw Do you guys have a prefer place to write technical designs down? Nothing serious, but it seems like the general theme is to create a plugin structure. I can write my thoughts down on this ticket, but some repos like to separates designs and tickets.

[mathiasrw]

That was a very thoughtful question - thank you for that.

We are (ab)using the issues to discuss and document things before they (normally) get into the wiki as more permanent information.

I have changed the title of this issue as it has ended up being more about plugin structure as you said. Feel free to place your content here.

[nickdeis]

@mathiasrw + @agershun

So I'm thinking we should start out with a build system that allows a user to easily create different "stacks" of parser + compiler + runtime. It would work something like bacon's build system where they would have to download the whole repo, but could pick and choose parsers either provided by the alasql repo or separate parsers/runtimes from other packages (working on an example for the latter).

However, it would have to be a bit more programmatic. We could just have a user install rollup, but we'd have to make it really, really easy for them to combine.

Also, we need to work on awesome taglines for the build tool. My contribution:

alasql-build: SQL, à la carte

[mathiasrw]

Omg omg omg - i looooooove it!!!

SQL, à la carte

Nice thinking...

I must admit I have no idea about how to structure a buildsystem like that in a good way (im sure I could fix someting up that I would have to rewrite 7 times)

I really like the solution they made on jquery mobile - check it out: http://jquerymobile.com/download-builder/ - its online and you get your file right away...

Its not that a buildsystem should not be made (they obviusly have a build system behind the website) - but I have a feeling it sounds a bit scary for some people - they just want to get started. I think it would be smart to brand 3 solutions

• Slim Very focused on size. Focus on select, join, group but no fancy input/output/rare SQL
• Full Has it all (like today)
• Tailor Only what you think you need

(or something like that - names can change)

So we can have slim and full will be on CDNs so people can link to them like now.

[nickdeis]

@mathiasrw

Slim Very focused on size. Focus on select, join, group but no fancy input/output/rare SQL

This was actually going to be my first example of a parser plugin :+1:

I completely agree with all of the above. Too often libraries adopt a plugin system and ditch their full builds. Alasql originally appealed to me because of it's ease of use, and I wouldn't want to take that away from new users. A good modular system is configurable, but has good defaults.

its online and you get your file right away

I've seen other libraries do this, and I've always really liked it. I made something similar for some colleagues who weren't familiar with npm.

I'm also working on a way of emitting AST to a serializable form so we can give the option of dropping the parser as per #94. Might not be able to do for a while. That might have to be a webpack/rollup plugin. Really the parser is the culprit for the large build size, and we should focus on slimming it down.

Adding to that list:

• WebSQL as per #94

I would have to rewrite 7 times

SoftwareEngineering :smile:

[mathiasrw]

:)

[mathiasrw]

Hi @nickdeis

Is there anything I can do to help out with https://github.com/nickdeis/alasql ? If you have pushed your latest changes I could follow your example and port more chuncks of code and pullrequest to your repo.

[nickdeis]

@mathiasrw Yeah, I'll clean up my local repo, drop that repo, clone this one, and drop the current ecma6 folder I have. Please feel free to be opinionated on the folder structure or the build structure. Might be worth it to create a new branch in this repo.

[mathiasrw]

Sounds perfect.

Made new branch https://github.com/agershun/alasql/tree/feature/plugin_structure

[nickdeis]

@mathiasrw & @agershun Wrote a very naive attempt at a build time transpiler here. Tried to make a rollup plugin, but alasql/sql weren't playing nice, so this one is for webpack. This is related #192 and all other tickets aiming to minimize the size. Currently using this for my own projects and it seems to work fine for inserts/updates/DDL. Can't due SELECTs, unless I don't understand how prepped statements work in alasql.

[mathiasrw]

interesting !

To see if I can cut out the clientside parser

I am sure @agershun can give some inputs if you put some more words to where you would like to cut the cake...

Happy new year :)

[agershun]

@nickdeis @mathiasrw I will join you on this topic after New Year! )))

[mathiasrw]

🎉🎊💥🎉

[elipalmer]

Has there been any recent movement on this ecma 6 refactor?

[mathiasrw]

Hi @elipalmer - thank you for reaching out.

In fact; there has! The last few weeks I have been cuddling with the code from 0.4.6 to mould it into typescript and ES6. The transition is not focused on a modular plugin code as this issue is addressing. This first step is focusing on untangling the (old school but efficient) "access everything via global scope" coding style of the library.

Lots of files still missing to be converted into the new feature. Basically, I run the test, halt on the first problem and convert / include files one step at a time.

I have just pushed the branch to https://github.com/agershun/alasql/tree/feature/rollup - this is work in progress - so wear your safety helmet and shoes if digging into the code after doing an npm test

[elipalmer]

This this awesome. If you need any help with this please feel free to ping me!