Plugin system

[ghost]

Why not have one piece of software that can serve as the Great Tree that supports all of these existing worlds?

I love the idea of Alda and I love this especially.

I understand this project is still very much a WIP, but a random sampling of issues has me wondering to what extent the syntax will support these "existing worlds".

The README says this:

Alda is designed in a way that equally favors aesthetics, flexibility and ease of use, with (eventual) support for the text-based creation of all manner of music: classical, popular, chiptune, electroacoustic, and more!

I'm a heavy Lilypond user, and while I acknowledge that its syntax can be a little arcane, it would be amazing if I could a) import my existing scores into Alda, and b) use the Lilypond syntax I've already learned to compose new scores.

Of course, not everyone will prefer Lilypond. Hence language modes. Perhaps Alda could support an optional language directive (defaulting to the Alda syntax when not specified) at the beginning of the source code (or specified as a flag on the CLI)?

[daveyarwood]

From the many conversations I've had with people today on HackerNews and Reddit, one major conclusion I drew is that being able to export Alda scores to LilyPond is probably be the single most in-demand feature we could implement. Importing LilyPond scores would probably be just as useful.

Syntax modes is a very interesting idea, but not one that I'm sure I'm interested in pursuing. The idea I have in mind is to keep the 2 languages separate, and provide ways to import and export Alda scores to/from LilyPond scores.

[crisptrutski]

Perhaps some sort of plugin system could come to the rescue here?

Imagining the ability to leverage boot --dependencies <plugin:version>, and an option to replace the parsing function/task used in the given context:

alda play --file blah.lilypond --plugin com.alda/lilypond-reader:0.1.0 --parser alda.lilypond/parse

[daveyarwood]

I've been kicking around the idea of a plugin system for a while now, I just haven't given much thought to how it might work. Leveraging boot is a good idea, I think. Maybe some combination of what you're describing and built-in "official" plugins that could be brought in using some kind of "require" syntax in the Alda file?

[crisptrutski]

Like your idea of bringing it into the syntax - altering the reader could work like Racket's #lang directive

[daveyarwood]

Yeah! My thoughts exactly.

[crisptrutski]

Not sure if composability from the CLI ala vanilla boot makes sense to you, but I quite like that idea too alda compose-some-serialism gimme-a-fugue-variation reggae-ify sheetify print-that-out-plz

[daveyarwood]

You just blew my mind. :boom:

[crisptrutski]

Was just thinking a bit more about this plugin story:

1. A built-in flag for overriding the grammar bnf gives some user happiness while plugin system is designed and implemented
2. Installing plugins could be something done using commands, and recorded in dotfiles - perhaps project.alda and ~/.alda/defaults.alda? Naming things.. :weary:
3. Installed plugins could have their dependencies loaded for every invocation, making new tasks and/or flags available. Perhaps even support overriding core tasks, with warnings

Putting all three ideas together in an example:

alda install-plugin lilypond -g
alda render --font-size 12 --file fantasia.alda --grammar reversed-octave-alda.bnf

[daveyarwood]

Another good use case for a plugin popped up in #122: a "pre-K mode" that treats everything as lowercase, so preschoolers could enter PIANO: C8 D E F G and it would still work. This brings up the idea of applying plugins to a REPL session. Basically, by using CLI options when starting a REPL, users could specify a plugin (or even a chain of plugins), and then those plugins would be applied to each line of text the user enters before parsing it. In the case of "pre-K mode," it would just "toLower" the whole line.

[crisptrutski]

Kinda thinking about writing an ASCII guitar tab plugin :)

[daveyarwood]

I had some thoughts today about how we could implement a plugin system.

I think the most crucial question at this stage is, what would an Alda plugin look like? I'm imagining a world where anybody can create an Alda plugin by making a GitHub repo, and anyone can use that plugin by including a --plugins flag to the alda play or alda repl tasks, including a comma-separated list of plugins to be applied to the score, in order. The job of a plugin would be to transform a string of text into another string of text, with the expected end result (after applying 1 or more plugins) being a string of valid Alda code.

Here's my idea:

• An Alda plugin is a GitHub repo containing literally any code, written in any language, but with some standard way to take input from STDIN and print the modified text to STDOUT. Maybe require plugin writers to include a script called plugin in the root path of the repo. A simple example would be a repo with only a single file, plugin, which is a Ruby script that upcases STDIN and prints it to STDOUT:

  #!/usr/bin/env ruby
  
  puts ARGF.read.upcase

• When given --plugins daveyarwood/upcase (or whatever the repo is called) as a CLI argument, Alda would clone that repo (or may look for it in a hidden directory somewhere), cd in and print the score text to it as STDIN and collect the output.
• A plugin could potentially be as complex as a whole project with the plugin script as an entrypoint. Boot-clj users could even write self-contained Boot scripts with Maven dependencies :)
• We could support "official" plugins by making them repos in the alda-lang org, and the alda-lang/ could be optionally left off when telling Alda which plugins to use, i.e. --plugins upcase

[daveyarwood]

Plugins will work best if they can be executed quickly, especially if being used in the Alda REPL.

To sidestep the issue of Clojure startup time for plugin authors who want to write them in Clojure, maybe we could offer a special alternate plugin format where Clojure code is read from a file (say, plugin.clj in the root of the plugin GitHub repo) and eval'd by the Alda process, which already has a Clojure run-time. There could be some convention, like defining a function called apply-clojure-plugin which takes one argument (a string) and returns a transformed string. Alda could then evaluate the contents of plugin.clj and run the score through apply-clojure-plugin before parsing it.

This could be a nice feature for Clojurists who want to write Alda plugins, even though they would be limited to the dependencies already loaded by Alda. I don't think that would impose much of a limitation, though -- plugin writers will have all of the core Clojure libraries available, as well as Instaparse, which is probably all that most people will need.

[daveyarwood]

It would also be nice if score-writers could specify which plugins their score requires by adding lines like:

# plugin upcase
# plugin daveyarwood/drum-tab

Alda's parser could pick up on these and fetch/apply the plugins, in order, before parsing the score.

[elydpg]

Wait, but isn't # used for short comment syntax?

[daveyarwood]

It is, but in this case, they could be "special comments" that we can look for before we even start parsing, so that we can use plugins to transform the score first. Technically, we don't even need these lines to be comments, since we can strip them out before we parse the score. It could just be:

plugin upcase
plugin daveyarwood/drum-tab

[daveyarwood]

Moved to https://github.com/alda-lang/alda-core/issues/2.