Use YAML and change merge_directory behavior

ucarion commented 10 years ago

I have two proposed changes:

We describe packages in YAML, not JavaScript.
We change the behavior of merge_directory to automatically exclude certain files (e.g. Toolbar plugin, Module Manager).
Changing from a Ruby DSL to YAML

Previously, we would describe packages using a Ruby DSL like so:

class MechJeb < Kmc::Package
  title 'MechJeb'
  url 'http://kerbal.curseforge.com/plugins/220221-mechjeb'

  def install
    merge_directory 'MechJeb2', into: 'GameData'
  end
end

The reason I used Ruby and not JSON or YAML to describe packages is because I expected that the download process for packages would be so inconsistent that a full-blown programming language would be necessary.

This initial worry was wrong. All we really need to install a package is merge_directory, and maybe we'll need a delete_directory someday. Describing packages with a full-blown, Turing-complete language is overkill and is overly complicated.

I therefore think we should use YAML. I think YAML is better for our purposes than JSON is because YAML is much less verbose.

Here's what I think a YAML package descriptor would look like:

# This is an example file that contains packages for B9 Aerospace and MechJeb.
#
# Just like before, one package file can contain multiple packages. Therefore,
# each package file contains a list of packages.
#
# Most of these parameters should be familiar; their behavior would be the same
# as they were with the Ruby DSL.
- title: B9 Aerospace Pack
  aliases: [b9, b9 aerospace]
  postrequisites: [b9 fix, b9 modzero]

  url: http://www.mediafire.com/download/o6cbe03iitggj1p/B9+Aerospace+Pack+R4.0c.zip

  # "install" has been changed. It should now point to a list of commands.
  # Commands recieve arrays of arguments.
  #
  # Just like before, the second argument to `merge_directory` is optional and
  # defaults to '.'
  #
  # See the MechJeb example for a case where the second argument to
  # `merge_directory` is used.
  install:
    - merge_directory: [GameData]
    - merge_directory: [Ships]

- title: MechJeb 2
  aliases: [mechjeb]

  url: http://kerbal.curseforge.com/plugins/220221-mechjeb

  # Note the two arguments to `merge_directory`.
  install:
    - merge_directory: [MechJeb2, GameData]

Have `merge_directory` auto-exclude certain files.

First of all, this is an option that can be opted out of. We can have a variant of merge_directory called merge_directory_no_exclude that does not have this behavior.

Many packages bundle Module Manager and/or the Toolbar plugin so that they work out of the box. Users of KMC do not want or need this feature; packages bundling other packages means KMC doesn't know about those bundled packages, and bundled packages might overwrite the versions of packages KMC does know about.

Instead of merging in bundled packages, KMC can instead selectively exclude bundled files (e.g. "ModuleManager*.dll" or "000_Toolbar") and instead have those bundled packages be listed as prerequisites for the package.

This auto-exclusion will not be comprehensive -- instead, merge_directory will simply choose not to copy files that look like they might be from a commonly-bundled package. At the moment, the only files I think we should auto-exclude are ModuleManager and the toolbar plugin.

Just because merge_directory will auto-exclude ModuleManager (for instance), a package still has to list ModuleManager as a prerequisite, otherwise KMC will never install ModuleManager.

/ping @kmc/owners @alampros Thoughts on these two proposals?

alampros commented 10 years ago

Sorry for the delay - was out of town for the weekend.

YAML: :thumbsup: I'm in favor. In the future, we might be able to easily use both JSON and YAML (one or the other per package), similar to how assemble uses both (via gray-matter).

merge_directory auto-exclude: :thumbsup: Good strategy. Over time, package authors will remove these bundled prerequisites anyway (hopefully).

Something else to think about related to this is providing the option for packages to specify scripts to execute at certain points in the process (such as post-merge or pre-test). npm provides this in the form of the scripts node in the package.json. Not sure how useful this would be for KMC, but it might be a good "catch all" option for weird mod installations.

ucarion commented 10 years ago

@alampros You're right -- we can add a post-merge callback if we ever find a highly-popular mod with a very specific install process that cannot be re-used for any other mod -- in any other case, I think adding more standard commands is a better option. But, following the principle of YAGNI, I'll add that feature when / if we need it.

I'm moving this week so I don't know how much time I'll have, but I'll try to get some of this stuff done.

kmc / kmc