Build team

[AndrewBelt]

Placeholder. Subscribe if you are interested in joining the Build team.

Workflow

TODO

Only the Build Team may change properties status and latestVersion in manifest files.

[AndrewBelt]

It seems that I am able to automate this process enough so that I can do this myself once a day. If you wanted to join this team, consider joining the Review Team #354, since the idea is basically the same except for making the final packages and uploading to the api.vcvrack.com server.

[phdsg]

@AndrewBelt do you plan on building at a fixed time every day?

ref: https://github.com/VCVRack/community/issues/405

[AndrewBelt]

No, it will be sporadic until I fully automate the process.

[AndrewBelt]

I'm close to actually making an automated build system running on Arch Linux

• mac: osxcross with MacOS 10.11 SDK
• win: mingw-w64-gcc from the AUR
• lin: system's GCC with voodoo to link to glibc 2.23.

Still working on win and lin. Still need to write scripts to look at the manifests for repoVersion, build the repo, upload the package, and set latestVersion = repoVersion and status = "available".

[cschol]

@AndrewBelt Cool! I tried osxcross, but never could get it to build. If you put all of this in a Ubuntu 16.04 Docker container you wouldn't even need the voodoo link for Linux. Or is this running in a VM?

[AndrewBelt]

osxcross works great for me right out of the box. No Docker needed. In fact, the multiarch/crossbuild Docker image doesn't work because its clang version is too old or something.

The AUR mingw build is failing locally. Still pushing it through each issue, but this might be too much time already. multiarch/crossbuild is really old and doesn't support all of C++11. Surely someone has made a pre-built mingw Docker build that is as modern as the one in MSYS2, right? Still need to look for one.

Linux build works fine through Docker's ubuntu:16.04 but haven't tested it more than a couple minutes. Main issue is that I have misplaced the list of Ubuntu dependencies (thought I committed it), so I have to add them one-by-one until I rebuild the list.

[cschol]

Weird that mingw build is failing. Didn't someone in another thread a while back say that he had a working Windows build in a Docker image? Actually, a PR.

Here is what I have in my notes from a clean Ubuntu 16.04 (Desktop) install:

libX11-dev
libgl-dev
xorg-dev
libasound-dev
libglu-dev
libgtk2.0-dev
libpulse-dev

[phdsg]

the untracked content after a distall build made me look at the repos' .gitignores. untracked content (usually a /dist or /build dir or a plugin.dll) only appears for submodules with "incomplete"_ .gitignore files in their repos.

i think it would be nice if everybody's repo had a .gitignore that includes the stuff created by using make dist.

[phdsg]

@AndrewBelt can you describe the details of the pull -> build procedure a little bit? i.e. what git commands do you use before launching the build script. i'd like to be able to reproduce and then fix the fetching errors we had a few times now, so only actual build errors can block the process.

[AndrewBelt]

I start the Docker daemon if it isn't running already and use git pull, git submodule update --init --recursive, and then python scripts/build_updates.py. I haven't published the Docker image I'm using, so it won't work out of the box.

[phdsg]

thx.

[AndrewBelt]

Closing because the build process will be pushing toward automation by me, rather than community participation.

[AndrewBelt]

Reopening to discuss the possibility of adding support for free closed plugins.

The goal is to supply builds as cheaply as possible while not sacrificing the level of trust VCV guarantees for free open plugins and commercial closed plugins. Thoughts?

[mdemanett]

What if V1 of the process is something like:

1. The developer MUST make the source available to Andrew and at least one member of the review team, by one of an approved list of methods, which might include: a. Grant read-only access to their private repo by user/pass or public ssh key. b. Zip or tarball download links, maybe password protected. c. Something else? A point here is that the method makes the source downloadable by script, so no things like emailing tarballs.

   2. Presumably whatever URLs and credentials come out of step 1, begin confidential, are not included in the manifests. Whoever has access would have to keep track of them somehow, and probably Andrew has to do a little work to make this machine-readable to the build script. (If the sharing method is only private git where access is granted by public ssh key, the repo URL can just go in the manifest, and there is nothing more to do. Keys can be shared by email.)

   3. Otherwise everything works as with the open source plugins. Review team reviews, Andrew builds.

Of course this entails a certain amount of trust. Andrew and review team agree to keep the sources confidential and really to only look at them as necessary to do the security and sanity checks we do for open source plugins. Developers agree that Andrew and review team are also developers and have a right to independently develop similar modules.

@Phal-anx @NikolaiVChr would something like this work for you?

[AndrewBelt]

We should probably make this method compatible with developers who wish to keep their code secret from even the build team, because for example most of the commercial plugin developers do this since they use proprietary libraries they can't share with me. The only way I can think of to put trust in the developers is if I have a government-issued ID on file (I've mentioned this before). It would be used in the 0.1% chance that someone tries to do something malicious, as a deterrent to push the chance to probably 0.01%.

[mdemanett]

Sounds reasonable to me.

[AndrewBelt]

That saves time for all of us because it allows a bypass of the Review and Build team. I'll draft up a template email and write instructions in the README for contacting me about free closed plugins.

[NikolaiVChr]

If I understood it right, it would work for me, yes.

[AndrewBelt]

Alright, a bypass is in place. https://github.com/VCVRack/community#addingupdating-your-plugins-build-for-closed-source-free-and-commercial-plugins

[MarcBoule]

Hi Andrew, regarding the build process, I was wondering if it should be the official procedure that devs always build their plugins against that latest released Rack SDK, as opposed to the Rack sources themselves. I think Sebastien-Bouffier also had an issue similar to mine, in which I'm using the latest Rack sources, which are incompatible with the latest released SDK. If I understand correctly, the build scripts all use the SDK. This would allow you to continue making changes to the main source, while not having to update the SDKs all the time since the devs would have to ensure their plugins build agaist the more stable SDK. Sebastien mentioned something about Rack 0.6.2 in his channel, and it got me thinking about this. I'm not necessarily advocating for this, as there are downsides to it, but I was wondering what the recommended method should be. Thanks as always! EDIT: just for completeness (and to show that I have searched before posting) :-), the community readme does not mention the SDK, and the plugin development manual at vcvrack.com mentions that to compile our plugins we can optionally use the SDK instead of building Rack (to save time).

[AndrewBelt]

Using either is fine. It's the choice of the developer to use the bleeding edge or the last beta release.

[AndrewBelt]

Closing, since the Build Team will be VCV eternally.