Choosing a software stack for the Exercism reboot

[ErikSchierboom]

In #113, plans are discussed for a reboot of Exercism. This will probably involve rewriting the front-end application. With a rewrite always comes one extremely important question: which stack to choose? Unfortunately, for client-side applications, the number of potential frameworks can be, well, staggering. There is Angular (1 and 2), React, Aurelia, Knockout, Ember, Vue, and much more. Recently, Matt Burgess tried to provide an unbiased overview of client-side frameworks.

To choose a client-side framework, there are several selection criteria, of which I feel the following are important (in random order):

• Ease of use: how easy is it create a client-side application with the framework? Can you just "jump in" and start building, or is there a steep learning curve? In other words, how many new concepts do you have to learn?
• Documentation: does it have good documentation? For a large-scale open-source project that will have lots of (new) developers, having good documentation is quite important.
• Adoption rate: how popular is the framework? Some frameworks might be technological marvels, but if nobody is using it, getting help will be so much more difficult and there is a greater risk of the framework being discontinued. Also, the more popular a framework, the greater the chance that people can just "jump in" and start helping.
• Complexity: in general, the framework should be as simple as possible. Some frameworks are quite complex in how they are structured, which makes them much harder to learn and use. Once again, as we want to have as many people contributing as possible, we ideally want a framework with low complexity.
• Feature set: obviously, the framework should serve the needs of our application, but I don't think we require anything out of the ordinary. It is worth noting that some frameworks are very minimal, and require adding libraries to fulfil some of the basic needs of an application. This usually increases the difficulty, although there might be tooling to help you with that.
• Performance: the performance should be good.
• Tooling: tooling-wise, there are huge differences here. There are several aspects here, one is IDE tooling: are there editors with built-in support for the framework? Another is the development tooling, does the framework have special command-line tooling to help you, or is there lots of manual work involved. Lastly, does the tooling help you with your development workflow? Does it allow for live- (hot-module) reloading, bundling, and such?
• Testability: how easy is it to write tests for the code using the framework? Is there special tooling involved? Is the framework mockable and are there examples on the web of how to structure / write the tests?

Perhaps we can have some discussion here on which frameworks people would like to work with and why?

[kytrinyx]

I think these are exactly the right concerns to consider.

[kotp]

The Client Side vs Server Side decision has apparently already been made?

[ErikSchierboom]

@kotp Well, not really I guess. Perhaps I should have made myself more clear and noted that I think using the specified client-side frameworks would be a good choice. It doesn't exclude server-side rendering, but it does imply more focus on client-side code. Perhaps we should be having the client-side vs server-side discussion at this point?

As a side note, almost all the criteria also hold for server-side frameworks. Shall I rephrase?

[kotp]

I just thought that the server side stuff would be decided first, what is going to be used to do the rewrite? Is it PHP, or Rails, Revel, or Phoenix or something else? Perhaps that doesn't even play into the decision for doing client side things...

[kytrinyx]

I don't think the server side language plays into the decision for the client side language, and I'd rather not discuss the backend language in this issue in order to keep this discussion on track.

[jtigger]

FWIW:

ThoughtWorks maintains a technology radar:

• https://www.thoughtworks.com/radar/languages-and-frameworks/react-js
• https://www.thoughtworks.com/radar/languages-and-frameworks/ember-js
• https://www.thoughtworks.com/radar/languages-and-frameworks/redux

[jtigger]

I've personally used Angular 1. I can safely recommend we do NOT use. :)

Scale:

1. excellent
2. good
3. fair
4. poor

• Ease of use: (poor) there's a bit of a learning curve; not horrible, but between the whole "digest loop", view model (aka "scope", and the idea that you're extending HTML with custom tags (aka "directives"); there's enough to require a couple of days of study.
• Documentation: (excellent) the team has done a great job at not just providing references but describing the key flows/processes in the framework. You can learn most of what you need to know from the docs. They do a good job of making it easy to find the docs fitting your exact version.
• Adoption rate: (excellent) from what I can tell, it's got the most history/support of the client-side SPA type frameworks; this means lots of resources.
• Complexity: (fair) there are a number of moving parts. The framework feels relatively heavy-weight when using it.
  • the $scope is a blackboard-style architecture ... think essentially global variables. The means of restriction (i.e. isolated scopes) means that data must be handed down through the hierarchy or through events or injected services: in practice, a pain, a mess or both.
  • the mechanism for binding works well for simple cases, but it takes major discipline to keep things straight as directives nest.
• Feature set: (good) AngularJS covers the bases and gives just a little bit more. For example, OOTB, there is a jQuery lite implementation that allows you to do most of what you'd want from that framework. Over time, I've had to pull in other "standard-library" augments like lodash.
• Performance: (good) with each version, it gets better. Certainly nothing we'll throw at it will be a problem (even in mobile browsers).
• Tooling: (poor: outdated - since 1.4)
  • I lost 2 solid weeks of my life trying to get Protractor (the official testing framework) working. I'm typically an average developer on my teams, so the pain I felt is likely typical.
  • last I checked, the browser extensions just didn't work. :(
• Testability: (good)
  • Angular 1 includes built-in testing features and examples for how to test each kind of component.
  • The framework itself provides hooks that support testing.

  • As I mentioned about Protractor is painful; but I've found component-level testing can easily be done in Jasmine or other JS frameworks in a way that avoids some of the time.

    (edit: added some rationale; originally, I just stated an opinion, which is useless in a discussion like this.) (edit: re-organized my input in terms of the measures that @ErikSchierboom laid out.) (edit: include section for testability.)

[ErikSchierboom]

Over the last couple of years, I've worked with several frameworks. Here's my take.

Scale:

1. excellent
2. good
3. fair
4. poor

Angular 1:

I have nothing to add to @jtigger's comments. We really shouldn't be using Angular 1, as it is old and not being updated.

Angular 2/4:

• Ease of use: (fair) it's a lot better than Angular 1, but there's still some learning curve getting to know the framework's unique vocabulary. And the template syntax is just plain weird, but you get used to it. They used to have lots of churning points, where there would be breaking changes even while already being in RC, but things have stabilized.
• Documentation: (excellent) the documentation is great, with a very nice tutorial that gradually introduces all features. As Angular 2 is built using TypeScript, most tooling will also display function/argument documentation when you hover over them.
• Adoption rate: (excellent) I think Angular 2 has been the fastest growing framework, with many people doing Angular 1 switching to Angular 2.
• Complexity: (fair) once again, it's much improved over Angular 1, but it still feels a bit frameworky, so there definitely is a bit of a ramp up. It also introduces case-sensitive HTML, which explicitly goes against the HTML spec, which might trip up people.
• Feature set: (good) there is a lot you get for free out of the box: routing, templating, forms handling, and much more. I also love the fact that they have a CLI which allows you to quickly get up to speed.
• Performance: (excellent) one of the main focus points of Angular 2 and it delivers admirably. They have a very advanced AOT compiler that can drastically cut the size of the output.
• Tooling: (excellent) being of the "big" frameworks, support in various IDE's is built-in. It also has the CLI which helps a lot. Lastly, being built on TypeScript makes for a very pleasant coding experience, as you get auto-completion and such in most IDE's.
• Testability (good): I haven't done much testing, but what I tried looked promising.

Aurelia:

• Ease of use: (excellent) this is perhaps Aurelia's best feature: it is just so brilliantly, elegantly designed. There is a great consistency in the concepts used, so you only need to understand relatively few concepts to become really productive. They try to comply with the latest standards, which means you can probably re-use existing knowledge. You can also choose to use ES6 or TypeScript, which in the latter case means you also get IntelliSense and the like.
• Documentation: (excellent) there is great documentation on almost every part of Aurelia, which is easy to follow and comes with code samples included.
• Adoption rate: (fair) I'm a bit in the dark here, but my feeling is that it is not yet one of the bigger players, although I only hear positive feedback from those that do use it.
• Complexity: (excellent) as Aurelia uses few concepts and builds on existing standards, I'd argue the complexity is not that high. You can quickly become productive.
• Feature set: (good) Aurelia got all the basics provided. Most websites can be built using just those default components.
• Performance: (excellent) the performance is very good, and will be more than sufficient for our goals.
• Tooling: (good) if you use TypeScript, the IDE have give you lots of great tooling. There is also an Aurelia CLI, but that is, while already useful, still undergoing some churn at the moment.
• Testability: (good) Aurelia can quite easily be tested, with even an end-to-end test being supplied in the default template.

Knockout:

• Ease of use: (fair) it is easy to get started with, but once you need more advanced features, things get significantly more difficult.
• Documentation: (good) the documentation is quite nice and gets you up to speed quickly.
• Adoption rate: (fair) there have been some big players using Knockout (e.g. the Azure management portal), but those are the exception I feel.
• Complexity: (fair) in Knockout, you almost always work through an intermediate layer (called observables), which can less intuitive debugging.
• Feature set: (poor) Knockout is basically just a model binding library, and does not provide routing and the like.
• Performance: (poor) one of Knockout's worst aspects is its performance, especially when dealing with large lists.
• Tooling: (poor) AFAIK, there really isn't much tooling.
• Testability: (good) testing is quite simple.

[ErikSchierboom]

Just came across this nice article on choosing a JavaScript framework.

[zachgersh]

@ErikSchierboom,

I had been talking to @jtigger a bit here in the office (since we work together) and felt like the selection criteria had one hole in it:

• testability

I know I have recently waded back into the world of frontend frameworks and have been slightly saddened that most frameworks devs talk about how many problems their framework solves but never mention (or even show examples) of how the whole thing should be tested.

Testing is so important and I believe it is EVEN more important for beginners looking to contribute to a project. A beginner, even if they don't TDD, should be able to play around with the tests or even just read them over to glean an understanding of what a component is supposed to be doing.

I am still struggling to adapt to the best practices for testing on the react project I am doing and I wish that testability had been given a consideration before the technology was adopted.

side note: seems like vue.js actually has docs on their site with testing examples (that is definitely a start)

[ErikSchierboom]

@zachgersh Great point! Totally forgot about that. I've updated my initial post and my response.

[zachgersh]

@ErikSchierboom - thank you for adding it! Could you add more more thing to the criteria for testability.

are there examples on the web of how to structure / write the tests

This one will be huge for when people don't have an example of what the tests are supposed to look like or have to troubleshoot an issue with a test.

[ErikSchierboom]

@zachgersh Done!

[jtigger]

For completeness' sake, I included it in the Angular 1 assessment.

[jtigger]

@zachgersh do you have enough experience with vue.js you'd feel comfortable enough to include an assessment, here?

[zachgersh]

@jtigger sorry for the delay - I unfortunately do not have enough knowledge of vue to be able to comment on its testability.

[arumoy-shome]

I am quite surprised that nobody has mentioned Elm. If you don't already know, Elm is a functional language that compiles down to Javascript. It's quickly becoming very popular due to it's delightful community and is being used in production by companies such as Pivotal and NoRedInk.

This video is a great place to start exploring the language.

[kytrinyx]

Also: http://exercism.io/languages/elm/about :-)

[joeljuca]

I'd rather prefer using more popular tools for the same reasons mentioned above (documentation, adoption, complexity, feature set, performance, tooling, testability), plus community. By using a more popular framework, chances are that peopl already know how to use it. Using a different language, framework, tooling, etc., is tough, since you'll need to learn best practices of the new language, framework and tools, plus understanding how to leverage its community, get help, etc., etc.

Regarding architecture, A decoupled system (web service oriented back-end with mobile/web client apps) might be better for us, since we could split a monolithic project into many, and manage them separately. If for some reason we are forced to use another language/framework/library/tool/etc. for the front-end or back-end we can do it by just respecting the HTTP interface (ideally, we would have an internal lib abstracting the HTTP communication).

When I think about Exercism as a product these arguments gets even stronger. I remember trying to solve an issue in the command line client, but I realized that it was written in Go, so I just gave up. But since Exercism is written Ruby, almost all its contributors are Ruby programmers and the language already have a pretty good community, tool set, etc., I couldn't imagine a good reason for it being written in Go (no need to discuss this matter, I'm just giving an example of how choosing a non popular language/framework might create barriers for contributions).

So, my #2cents: keep Ruby on Rails as back-end - since it's a rock solid platform and is still evolving very well, and choose a front-end frameworks for desktop/front-end/mobile apps. I would use React, since its ecosystem is really good, is growing very fast, everyone knows React and there's even solutions like React Native for mobile development.

[kytrinyx]

Sidebar:

I couldn't imagine a good reason for it being written in Go

The CLI was originally written in Ruby, which was great when there was only one language track. The problem arose when we had people doing languages like Haskell: they needed to install an entire Ruby development environment... just so that they could do Haskell exercises. Go lets us distribute a single binary with no runtime dependencies. You stick it in your PATH and it works. On all the operating systems that we support.

Also: many contributors to Exercism do not know Ruby. Many know Go.

We are trying (and doing a better job of) choosing more common technologies that are better documented and are more likely to attract contributors, while also making choices about the best tool for the job. For the CLI, Go was a good choice at the time, and remains a good choice. For the website Ruby is a good choice, but Sinatra (the current choice) is not. We will use Rails in the next iteration, because the documentation is better, and the standards and conventions are stronger.

For JavaScript it's truly difficult to choose a "standard"—there's really no such thing in the JavaScript world. For anything. And yet, there are certainly things which have a longer history, better documentation, and which have good trade-offs for some situations and not others.

Our final choice will likely depend on both how well-established the technology is, and how much JavaScript we need and what we need it for.

[joeljuca]

@kytrinyx thank you very much for such a great explanation!

[ErikSchierboom]

@kytrinyx At what point in the reboot are we at the moment? Are we anywhere near choosing a software stack?

[iHiD]

Hi @ErikSchierboom. We (Thalamus + @kytrinyx) are currently building out a prototype based on everything we've been discussing from the last few months. The prototype is a straight-forward Rails app (with the API ported into it) and just using standard HTML for the front end. No front-end framework and minimal JS in general. Our aim is to have that launched in the next few weeks. At that point, we'll push it out to the world for all the feedback possible and then either decide it's great and that it should be the permanent solution, or the we'll all decide that it needs lots of things doing and maybe would be better if it was developed using some other tech stack.

The key aim of this process is for us to understand whether all of the concepts and decisions we've made make sense, and whether the UI that we're designing "works" from a product and UX POV. The specifics of the tech that powers it is something I feel less concerned about, so I'm going for established harder-to-make-bugs-in tech, which Rails fits for me.

Does that make sense?

[ErikSchierboom]

@iHiD That makes perfect sense! Looking forward to seeing the results.

[kytrinyx]

We ended up not using a whole lot of JavaScript at all—if this changes, then I think we should reopen this issue.

Thanks everyone for weighing in. This whole v2 prototype process has so many moving parts, it's really great when the community is able to grab one and run with it.