SPIKE: determine backend language

[emmahyde]

Blocked by:

• 1

• 2

Acceptance Criteria:

• create cards tracking application architecture implementation, i.e. getting the boilerplate application up and running, tracking docker work, integrating mkvtoolnix, etc.

[Shaka-n]

Requirements: speed, parallelism, and a suitable testing framework

[Shaka-n]

It seems that only some Ruby Interpreters support multi-threading. CRuby has concurrency (async), but the Global Interpreter Lock prevents multiple threads from running in most cases. JRuby's interpreter doesn't have this limitation, but I don't know what other tradeoffs that would come with. If we were to use Ruby to achieve MVP using concurrency instead of parallelism, we would probably need to utilize the fork command or Promise objects.

[emmahyde]

The language I would most like to learn in order to support concurrency/parallelism is Elixir, which supports both concurrency and parallelism out of the box. Elixir is inspired by Ruby, but is a functional language and is mostly only similar to Ruby in its syntax/appearance. There would be a learning curve here, but considering the fact that we're dealing with files and other functional data structures - as well as our lack of need for a database or anything that could be considered an "object" - might make Elixir our best bet. Elixir has an irb equivalent (iex) and also has a (very well received) framework called Phoenix that has a very robust frontend experience. Lots of people tout it as possessing one of the best javascript-competitive frontend development experiences. If we do ultimately need a db, Phoenix also offers a comparable ActiveRecord ORM replacement that is equally well-received. A con of this approach is that Elixir is that neither of us are familiar with it in a meaningful way, a lack of resources, and the fact that it is in its early-ish life and has a vastly inferior package library to either Ruby or Python, which means in many cases that you will need to write your own wrappers, et cetera.

The language I would be able to write in most efficiently, and would get us from A to B the fastest, is Ruby. Ruby supports concurrency in very rudimentary stages, but not parallelism. It would be ideal if we could write this particular application in Ruby 3.0, but unfortunately there is a major lack of support for that right now. We would certainly be able to prototype the quickest and we are both already familiar with and love Ruby, but I worry that we'd be blocking ourselves off from improvement by choosing it for this project.

The language that has the most external support for what we're trying to do is most likely Python, it has the community behind it, massive and active package catalog, highly popular, good to be familiar with, and functions natively on all operating systems. We're dockerizing, so this last point isn't a massive deal, but Python brings us pretty close to the lower levels of the operating system & has an incredible wealth of tried-and-tested ways to interact with bash and the system, without forcing us to handle pointers or garbage collection. I'm not entirely clear on Python's native support for concurrency/parallelism, but there is a vast ocean of packages that would take the concern off of our hands. Cons, I don't love Python, if we have to learn something I'd be more interested in learning Elixir, horrible test writing experience at least with pytest, unfamiliarity with web frameworks and frontend integration & generally just kind of feels like it was created by someone who loves Windows.

Given all of the above, I'd say the language with the biggest potential payoff would be Elixir, provided that we can wrap our heads around the functional approach without blowing our brains out, as well as ensuring it has adequate ability to interact with bash and the file system. I'd say the action item for this is to create some cards dedicated to us learning into Elixir a bit and then to reassess if its too big of an ask and just go with Ruby. I think choosing Ruby is what I'd most like to do, but I can't deny that its uniquely unfit for this project, especially given that we do not have a use case to take advantage of ActiveRecord ORM (no db) and Rails.

[Shaka-n]

This all makes perfect sense to me. Seeing as you know best as to what we're getting ourselves into, I defer to you. But allow me to make a further argument for Elixir. I consider our priorities to be:

1. Make something cool/useful
2. Learn something new
3. Have fun

Considering that we both organically want to learn Elixir, that it is well suited to the task, and that we are not necessarily constrained by hard deadlines, I think that it is our best candidate.

While we will learn things if we use Ruby, in my view rapid prototyping does not provide as much benefit to us or the ultimate product as Elixir might. Learning Python would be the compromise, but from what you're saying it just sounds like it would be less fun.

I agree that looking into Elixir is the next step, and it may prove to be too much of an ask, but I wanted to lay out my thoughts anyway.

[emmahyde]

Done! Elixir it is.