Introducing kdmodules

[tetsuo]

Hey kd people,

I've spent most of the last week working on kd framework and just came out to introduce you to kdmodules, a very humble attempt of my own to modularize this beauty for future generations :)

I actually have applied for a job interview in Koding last week and seems like it didn't work out as expected :) But anyhow it was fun to dive into the this beast and see what you guys are up to. It kept me busy for a while and I didn't want to throw away this effort.

Following is my original proposition to @devrim. I'd love to read your opinions one way or another, and kudos- keep up the good work :)

a guide to kdmodules

This document covers a quick introduction to kdmodules and discusses its application to workflows for building modular web applications.

introduction

kdmodules is an ongoing project that aims at modularizing the kd framework.

what is kd framework?

kd framework, an open source community project led by Koding, introduces a nice set of ui widgets and serves as the base library for kdapps.

It resembles the early stages of the mighty Google Closure Library. One major difference is that it is written in CoffeScript with object-oriented paradigm in mind all the way down to its very core; thus allowing easier abstraction over classes and brings joyful tears to our eyes which already lost half of its sight due to long hours of struggling with inheritance, prototype chains and etc. in javascript, infamously the bad parts.

But most importantly it is a thin integration suite under the hood and provides the necessary tools to integrate your applications into existing Koding ecosystem. It is one of the many solutions to build beautiful things on Koding if not the easiest.

kd, the bad parts

We have seen the good parts, now let's talk about the other way around.

It's been roughly a week since I have introduced myself to kd. Not forgetting the huge effort that took from many many well programmers and designers to build these crazy collection of widgets; if I may put my two cents in, that among not being well-tested and documented it lacks the structure to support modular applications of the future web, so to speak.

the future of the web

We, as front end engineers, we have suffered a lot. We have spent many precious hours building workflows, managing assets, keeping our 3rd party libs up to date, destroying workfows, fixing compatibility errors, building workflows again, fixing cross browser issues and not being able to test anything before building something useful, if not already got fired.

You see, it was like zombie apocalypse back then, we hardly had time to breathe.

A combination of Grunt+Bower, Yeoman, RequireJS and several others have eased the pain a bit, but unfortunately they introduced a far more complexity than needs to be.

Hopefully with the introduction of Web Components, this will end. We will just whip up a main.js, require some stuff, emit some events here and there, and bam ready! your next ground-breaking idea is live on the web!

Actually I lied, it won't end; because most of the libraries around are not quite compatible with modularity yet. Anyways, we should not worry about tomorrow's problems until tomorrow, each day has enough trouble of its own already. But it's never been the worst idea to start building tomorrow's strength today.

back to today and the future of kd

We don't have our Web Components spec ready yet. But there are alternatives, most notably Component.

Component community put a great deal of effort into this issue and built one of the most-awesome solutions to tackle building modular applications on the web. (See tj's blog post, smalljs site offers some beginner introduction too.)

And there are others inspired by Component, Duo, normalize, webmodules, npm-dom...

Frankly kd framework doesn't quite fit in with this emerging philosophy of modular web application development.

And this is where kdmodules steps in to the scene.

brace yourselves, modularity is coming!

Long story short, kdmodules organization on Github acts as a registry for all of the libraries within kd framework.

Paired with kd.js, kdmodules builder tool, it makes extremely easy to build kd applications.

Let's see how easy it is.

getting started with kd.js

With npm do:

npm i kd.js -g

This will install kd command globally, if you don't want to install globally just remove the -g flag.

kd -h shows some help on usage. But actually it is easier than you think.

You build kd applications like this:

kd index.coffee

kd uses browserify by default. However all kdmodules are compatible with component spec, and therefore kd provides an option to build your modules with component-builder.

kd -c

benefits over using big ol' kd

• Semantic versioning (npm & component)
• Allows incremental builds with component & watchify
• Use any component builder plugin or browserify transform, thus use any css preprocessors, templating languages you'd like
• Easy maintenance, no need for huge testing stuff, coverage libraries and stuff
• UNIX philosophy in mind, "Write programs that do one thing and do it well"
• Supports npm/github/bitbucket
• Ability to require over 1600 components

  examples

For browserify builds, install the kdmodules you need with npm first:

npm i kd-core kd-header --save

And start coding right away:

core = require 'kd-core'
header = require 'kd-header'

main = new core.KDView
main.appendToDomBody()

foo = new header.KDHeaderView
  type: 'big',
  title: 'bar'

main.addSubView foo

And build it with kd:

kd index.coffee -o output.coffee

If you choose the component path, you won't even have to install a single library manually.

Just make a component.json, something like:

{
  "name": "some-awesome-site-name",
  "version": "0.1.0",
  "dependencies": {
    "kdmodules/core": "^0.1.3"
    "kdmodules/header": "^0.1.2"
  },
  "scripts": [
    "index.coffee",
    "*.js"
  ],
  "styles": [
    "whatever.sass",
    "css-preprocessor.styl",
    "you-like.less",
    "or-all-combined.css"
  ]
}

roadmap

• modularize all the way
• bring stylesheets into the party, allow devs to just fork an official kdmodule repo and use a custom theme
• cleanup global namespace pollution
• tests
• remove jquery, devs should install it if they need
• api consistency
• improve kd.js so it can uglify
• support sourcemaps and all that nifty debugging stuff

  notes

https://github.com/tetsuo/kd.js/issues/1

[sinan]

@tetsuo thanks for starting this. You've just done a good part of a migration plan, what we were planning to do here. There are some style issues but not major stuff. My and @usirin's comments will follow.

[gokmen]

Great start @tetsuo :+1:

[tetsuo]

I am adding you guys to kdmodules org. And I can add you as owners to npm packages if you tell me your usernames.

Some detailed notes I've taken for migration are as follows:

• My intention was to keep 0.1.x branch of all kdmodules intact and use it as a reference point since it is the original codebase; but most importantly for the fact that they are not fitting well into any modular package manager and builders around.
• That's the reason why I translated all of the file names to their exported class names using the mapping on entry.coffee In fact that caused some trouble (https://github.com/componentjs/builder2.js/pull/85) since component is not case-sensitive. But whatever, as I said 0.1.x is quite experimental and should be seen as a stopgap with documentation and readable file names and all that bling.
• One another advantage of keeping old stuff in 0.1.x is that both component and npm support semantic versioning. So you can work on the next generation in 0.2.x branch and still be able to maintain backwards compatibility without any hassles.
• and kdmodules org should only contain modules that are part of the inheritance chain, that must be the spirit I guess.

As for 0.2.x branch, turning this giant into tiny components is a fairly straightforward task: Encapsulate everything in its own space

These are my notes:

• Break core module into chunks. There shouldn't be any core module.
• Actually break everything into modules, even if it is a tiny function like utils.curry: if it is shared than it deserves to be factored out into a repo of its own.
• Remove global namespace (window) pollution, that breaks everything. Let users name the exported classes however they need. Or you can implement something like Closure library goog. space, BUT just for development.
• Remove circular dependencies. That introduces the main problem of core module. core depends on notifications, notifications depends on forms, that depends on list and buttons, that depends on core again and vice-versa, and all of a sudden you end up building almost half of the library for just a single Header. Without compression HeaderView example above costs 500kb, in theory it should be around 5-6kb.
• Removing circular dependencies, and global names will also introduce the ability to use destructuring assignment goodie of CoffeeScript. Right now require ordering must be very specific in order to run this.
• Make polyfills module optional
• Remove jQuery, because it sucks
• For themes, it is really not easy to make them work with package managers. I think currently the best option is to have a separate repo for them. If a user wants to use a custom theme, just tell him/her to fork that up.
• Why so singletons? If it is Koding infrastructure related, a better alternative is to whip up some vm and pass require into that context. If it is not; both browserify/node's and component's require implementations are more than enough to take any care of singleton needs, just return an instance, it will be cached.

That's most of it guys. I'm afraid I will not be able to keep up, but anyhow hope that roadmap helps you on your way :) This library deserves to be future-proof, tested and documented, there is some huge effort in it.

[usirin]

@tetsuo I really appreciate the effort you put into KD, and all of the kind words you wrote!!

Like @sinan mentioned already, we started the kdframework organization for the exact same things you described.

Like you said, we are planning to export all of the objects outside of one single repository, and make them standalone modules that depends each other via package.json includes. That's all.

Separating them into single repositories is exactly what we are planning to do, but it's not enough to be honest. But you already mentioned this with the idea of 0.2.x branch.

KDFramework took us here without thinking about any other framework, and we are really proud of it, but like you said, it requires a touch to make it contemporary. But these days as you can imagine that amount of work requires full attention and focus, but till this day -and probably a little bit longer- KD hasn't been our primary focus, this is basically a project that supports Koding internals, and we neither have time for it nor we like to do refactoring just the sake of refactoring.

So, kdframework organization plays a really important role in this. Before that, we were always developing KD according to Koding requirements. But this kdframework organization will not depend on Koding, but vice-versa.

This is why any effort community puts in KD is really important, your points align with ours very well, implementation is only a matter of time. Thanks for the suggestions.

[szkl]

Actually break everything into modules, even if it is a tiny function like utils.curry: if it is shared than it deserves to be factored out into a repo of its own.

Separating them into single repositories is exactly what we are planning to do

This would just make inter-module dependency and version management harder for basically no practical gain I can think of.

[tetsuo]

NOW THAT'S MASSIVE, just talked to @sinan and guess what guys! :dancer:

I'm incredibly excited to announce that I will be joining the Koding team working on the future of some awesomeness :) https://www.youtube.com/watch?v=UgV7hnjoyt0

[devrim]

dat video.

congratz!

On Nov 27, 2014, at 15:40, Onur Gunduz notifications@github.com wrote:

NOW THAT'S MASSIVE, just talked to @sinan and guess what guys!

I'm incredibly excited to announce that I will be joining the Koding team working on the future of some awesomeness :) https://www.youtube.com/watch?v=UgV7hnjoyt0

— Reply to this email directly or view it on GitHub.

[tetsuo]

@szkl

Thinking of the current code base, what you say is reasonable.

But if you write modules in a truly decoupled way (which in fact is not-that-radical at all) it is quite the opposite.

Keeping everything separate in npm registry and/or Github then introduces some first-class version management and interoperability. And now if you have a custom builder of your own (which you don't really have to actually, since all modules are compatible with browserify and/or component) dependency management is a piece of cake. It is just a minute of editing a component.json file or typing npm i package name@version --save

This talk explains better than I do now: https://www.youtube.com/watch?v=DCQNm6yiZh0

@devrim Thank you :)