Isolate Alda components into separate projects

[daveyarwood]

Right now, a handful of different things are lumped into this one repo (alda-lang/alda):

• The "Alda core library" which includes...
  • alda.parser: parses Alda code into Clojure code (alda.lisp DSL)
  • alda.lisp: the score environment that allows you to evaluate a score S-expression and end up with a Clojure map of data representing a score
• alda.sound: The Clojure implementation of the sound engine, which uses the JVM (javax.sound.midi, eventually JSyn for waveform synthesis) to interpret the score map and play the score.
• alda.server: A Clojure HTTP server application that starts an Alda sound engine and score environment and exposes endpoints allowing clients to interact with the score and (if the server is running on the same computer or another computer within hearing distance) hear the results.
• The client: A Java command-line client for interacting with Alda servers.

I wonder if it might be a better idea to split these things out into their own projects within the alda-lang org, so we can focus on developing them separately. We can deploy artifacts to Clojars, and the other projects can pull them down as dependencies.

A nice benefit of this is that Clojure programmers can play around with using Alda as a library without needing to pull in the sound-making dependencies if they don't want to. The "core library" component of Alda ought to be very light. I would personally be interested in playing around with the Alda core library + Overtone as a sound generator, just for fun.

This will also make it easier to experiment with alternate implementations (ClojureScript, etc.) of the server, client, sound engine, etc.

Usefully, waffle.io allows you to track multiple issues on one board, so we can still get a nice "big picture" visual of the development of the Alda project as a whole.

It will probably be a while before I get around to making this change, but until then, I thought I'd get this idea out there and let it marinate. I'm very curious to hear other developers' thoughts on this.

[daveyarwood]

I'm making great progress on this so far. I think it should be done soon!

So far:

• I've extracted out the following components into separate repos:

  • alda-client-java: contains just the Java client code
  • alda-server-clj: contains just the server and worker Clojure code
  • alda-core: everything else -- alda.parser, alda.lisp, alda.sound, alda.now and alda.repl

  I haven't yet pushed these repos to GitHub, but will do it soon.

• This repo (alda-lang/alda) will continue to exist and function as the "landing page" for the Alda project on GitHub. It will include docs, the same user-friendly README we have now, the script to install the FluidR3 soundfont, and releases.

  So, the end user experience will be exactly the same as it is now, it's just that the source code for all the components will now live in separate, focused repos.

• I have a branch of this repo ready that has all the source code stripped out and moved to the repos listed above. There is no source code, only a build.boot that depends on the subprojects and can be used to build releases.
• The new release process will be:
  • Deploy new versions of the changed subproject(s) to Clojars. For example, if there are changes to alda-core, then we would deploy a new version of alda-core to Clojars.
  • Update the dependencies for the changed subproject(s) in the "main project" (alda-lang/alda) build.boot file.
  • Run boot build -o $output_directory like we do now.

---

TODO:

• [x] Main project README "Contributing" section: mention the breakdown of repos in GitHub and refer to CONTRIBUTING.md for more detail.
• [x] Main project CONTRIBUTING.md: more detail about the above
• [x] Development guide: pull out component-specific development instructions and include them in each subproject repo as either a separate development guide file or a "Development" section in the README if it's brief enough.
• [x] Consider if there is any development-specific info in the main repo docs that would make more sense to include as part of the README, etc. in the subproject repos.

• [x] See if we can break alda-core up further.

  There is currently a lot of stuff in the alda-core repo. I realized that the alda.lisp and alda.parser components are fairly intertwined, so at the minimum alda-core needs to contain both of those things.

  • [x] Can we break alda.sound out into an alda-sound-engine-clj repo? It doesn't depend on alda.parser or alda.lisp; its entrypoint is basically a play! function that expects a fully realized Alda score map.
  • [x] Figure out what to do with alda.now. It is used by alda.repl and alda.worker to play scores. And I also think alda.now is a good thing to provide itself as a library, because it allows you to experiment with Alda in a Clojure REPL, and to use Alda in Clojure programs.

  alda.now depends on both alda.sound and alda.lisp -- it's sort of a user-friendly bridge between them. I'm thinking maybe it should be a part of the alda.sound project, which could have an alda-core development-only dependency.

  So then, to use Alda in a Clojure project, you would want to include the alda/core and alda/sound-engine-clj dependencies, and use the alda.lisp and alda.now namespaces in your code like you do now.

  I think alda.sound could still function without alda.lisp potentially, e.g. if somebody wanted to implement an alternate parser/compiler and pass the result to alda.sound/play!.

  • [x] alda.repl will eventually be deprecated and its functionality replaced/improved in the Java client code.

  For now, I think we can extract it into an alda-repl-clj repo with dev dependencies on alda-core and alda-sound-engine-clj.

  The main project can then include alda/repl-clj as a dependency until we implement the REPL functionality in the Alda client.

  At that point, we can remove the alda/repl-clj dependency from the main project and mark the alda-repl-clj repo as deprecated.

• [x] Make subproject repos publically available on GitHub.
• [x] Update our waffle.io board to track issues and PRs from all of the subprojects as well as the main project.
• [x] Go through the issues in the main project repo and "move" them to the appropriate subproject repos. In most cases we can create a new issue in the subproject repo with a summary and a link back to the main repo issue for context, then close the main repo issue.
• [x] Open a visible "READ THIS BEFORE FILING AN ISSUE" issue in the main repo with a helpful explanation about the subproject repos that will make it clear which repo to use for the issue you want to file. Mention that if you're still not sure, you can just file the issue here and it will be reassigned by the Alda team.

[elydpg]

3 more issues to go!

[elydpg]

Also I'd imagine that releases would still be pushed and available for download here; on the central project.

[daveyarwood]

Yep, that's exactly what we're doing!

The "main repo" (i.e. this one) now functions as a landing page, a place to find documentation, and a place to get the latest release.

[elydpg]

Would #50 go under main or one of the other repos?

[daveyarwood]

I was wondering about that... it doesn't really fit under any of the other repos.

The closest match would be the website repo. I'll probably move it there.

[daveyarwood]

Done!! 🎆 🎆 🎆