Migration to v1

[AndrewBelt]

Time to start discussing the migration to Rack v1 and the changes required in this repo!

1. When the v1 API is stabilized, we should make a new branch v1 in this repo. Yes, there's already a v1 branch, but it was foolishly named years ago, so we'll just remove it.
2. Instead of writing/editing manifest files ourselves, we should only approve them from plugin developers. We should help them out if needed. JSON is not a skill that everyone has, and we have certain guidelines for some of the properties. Luckily, testing your plugin with Rack already checks most of the plugin.json like invalid JSON or some bad properties. When approved, we'll simply add their repo to repos/, run a script that copies repos/MyPlugin/plugin.json to manifests/MyPlugin.json, and commit.
3. Later, after Rack has a command line flag to dump module screenshots, we should add them to this repo in screenshots/. Err, maybe not. An envelope calculation says that it'll be over 100MB if screenshots are at 200% scale (~100KB/screenshot * 1000 modules). This might still be okay though.
4. Because library manifests will be identical to repo manifests, the following properties need to be rethought.
   • repoVersion: We can just read repos/MyPlugin/plugin.json version for this, no longer needed
   • latestVersion: This will just be version since it's in the plugin.json format.
   • productId: This will be handled via slugs by the server. It can be removed.
   • buildTimestamp: Not sure where we should put this. Ooo, I should just generate it on the server by reading the package file timestamp.
   • status: Can be computed by the server as well.
   • Okay, so all of these are handled. Fantastic.
5. The server API will handle requests for both v1 and v0.6 plugins (at least for a while), via a new API path or query.

All other operations will probably be handled the same way. Can anyone think of anything I'm missing?

[AndrewBelt]

I think we should adopt a strict policy disallowing slug changes for plugins and modules. I can understand the OCD of plugin developers wanting to change their slug when their plugins' name, scope, or purpose changes (e.g. from AndrewsCoolVco to AndrewsCoolCollection), so if they don't have a hard reason (i.e. us) to keep the old slugs, they will make the change, which has no advantages other than satisfying perfectionism, and has major disadvantages to the users and the repuation of VCV Rack as a "stable" platform.

[cschol]

OK. I think you should communicate this to the developers in the forum and on Facebook.

I will keep an eye out for slug changes during the review.

[AndrewBelt]

I've recently discovered how powerful jq is for parsing manifests. For example:

jq '(.brand // .name) + " " + .modules[].slug' manifests/*

will display the full names of all modules in the VCV Library.

I like using Python for scripts that I save to a file, but maybe this will come in handy for hacky one-time tasks.

[cschol]

A little script I use to check if the plugin version was updated (assumes that you have pulled the repo to the author's desired state): https://gist.github.com/cschol/2fed47fcf44552b321b98c6e980fa36d

Returns non-zero exit code when the version is the same.

cschol@cschol-dev MINGW64 ~/src
$ ./check-plugin-version.sh $PWD/library ImpromptuModular
Old version: 1.0.1
New version: 1.1.0

~I'll add it to my Python manifest validator script later.~ Added to Python script.

[cschol]

Not sure if any of these fit into the library repo itself, but I figured we can see if anyone else can get use out of them: https://github.com/cschol/vcv-library-tools

Both are specific to my workflow of integration, but I am open to feedback on how to generalize.

[AndrewBelt]

You can put them in scripts/ if you like, along with the README referencing those files.

[cschol]

If I find other slugs that have been changed and weren't caught (e.g. LindenberResearch has two filters that changed slugs), do we have them change them back?

[AndrewBelt]

There is a tradeoff as I explain here. https://community.vcvrack.com/t/autodafe-plugins-updating-all-the-time/4369/14 I'd say if the slug has been changed more than 2-4 days ago, keep it changed. But that's just my educated guess for what's best.

[cschol]

OK. That makes sense. It was in the update 3 days ago.

I see now what was going on: the plugin had whitespace in the slugs and he replaced it with "_".

[cschol]

I built a collection of all v0.6 slugs with a fancy sed regex and dropped it in a file. Now the vcv-manifest-validator.py will look in that file for the plugin/module slugs and flag them if not found:

Example:

[LindenbergResearch] Issues found in `plugin.json`:

Missing key: brand
Missing key: pluginUrl
Missing key: manualUrl
TYPE35_VCF: module slug flagged
DIODE_VCF: module slug flagged
BlankPanel_01: module slug flagged
BlankPanel_Wood: module slug flagged
BlankPanel_02: module slug flagged
BlankPanel_Small: module slug flagged
MS20_VCF: module slug flagged
Westcoast_VCS: module slug flagged

Other relevant example:

[rjmodules] Issues found in `plugin.json`:

rjmodules: plugin slug flagged
Acid: module slug flagged
EssEff: module slug flagged
uQuant: module slug flagged
MetaKnob: module slug flagged
ReplayKnob: module slug flagged
Pluck: module slug flagged

Unfortunately, it will have false positives with new 1.0 modules, but that's manageable.

[cschol]

@AndrewBelt How are you triggering the new builds? I added ByteBeatMachine last night and then realized that there was a crash in the module browser. I moved it to "Needs Repair" state and thought it would not get build, but it did. Same with FrozenWasteland, which was added, but later moved back to "Needs Review" due to an update.

[AndrewBelt]

A build will be made iff repos/<MyPlugin>/plugin.json is different than manifests/<MyPlugin>.json. It doesn't mess with the GitHub Issues API, that'd be too complicated.

[cschol]

OK. I didn't know how automated this was. I thought you actually looked at the project board and built what's in the queue.

This means that I should have reverted both commits in the repo to prevent them from being built, which is fine, I think.

Hadnt even thought about using the Github Issues API, which is interesting.

Other question: is there a way for me (via commits to the repo) to disable a broken plugin from the Plugin Manager? ByteBeatMachine is crashing right now. Would removing the manifest from manifests disable it?

[Coirt]

@cschol Updating the manifest for ByteBeatMachine to include "disable": true should disable it.

{
  "slug": "ByteBeatMachine",
  "name": "ByteBeatMachine",
  "version": "1.0.2",
  "license": "BSD-3-Clause",
  "brand": "ВОКОИТЕР",
  "author": "bokontep",
  "authorEmail": "bokontep@gmail.com",
  "authorUrl": "https://bokontep.blogspot.gr",
  "pluginUrl": "https://github.com/bokontep/ByteBeatMachine",
  "manualUrl": "http://bokontep.blogspot.com/2018/05/bokontepbytebeatmachine-plugin-for.html",
  "sourceUrl": "https://github.com/bokontep/ByteBeatMachine",
  "donateUrl": "https://www.paypal.me/AnastasiosKleisas",
  "modules": [
    {
      "slug": "ModuleByteBeatMachine",
      "name": "ModuleByteBeatMachine",
      "description": "ByteBeat Oscillator",
      "tags": [
        "VCO",
        "Utility"
      ],
      "disabled": true
    }
  ]
}

[cschol]

I get that, but that is not how the integration works. The manifest is owned by the author. This will not prevent the faulty plugin from being available in the plugin manager.

[Coirt]

Its a temporary solution until the plugin can be fixed. I'd recommend that to the author but I do not have the authority. Probably would be best to get an update for the plugin with the above manifest ASAP so everyone who is effected is not crashing randomly, not knowing why. It is either that or damage the Authors/Racks reputation.

Could you not just fork and redirect to the forked manifest?

[cschol]

No. The author needs to fix it. Until then we should remove it from the plugin manager. I don't know the proper way to do that hence my question above to @AndrewBelt.

[Coirt]

I'm aware that the author needs to fixed it, while it remains broken the cleanest solution in my opinion, is not to remove it but to disable it, on the next update when it is fixed the version can be incremented again and enabled. It is one line of code as opposed to removing all the builds from the server, removing the builds is not going to remove the plugin on everyone's machines, it will still crash Rack.

[AndrewBelt]

The tag data structure has been rewritten and expanded. Use this to update your validation scripts. https://github.com/VCVRack/Rack/blob/v1/src/tag.cpp https://github.com/VCVRack/Rack/blob/v1/include/tag.hpp

[cschol]

Thanks. Script is updated.

[AndrewBelt]

I've migrated old (<1.*) plugins to the v1 branch. This cleans up a lot on my end because I was previously merging the two branches on the API server. You might want to run your validation scripts on it, to fix issues like license strings. You can edit them directly, since the developers of old plugins don't maintain manifest files.

[AndrewBelt]

Closing since this issue has served its purpose, and the VCV Community forum is a better place for discussions with users.