steinbergmedia / vst3sdk

VST 3 Plug-In SDK
Other
1.65k stars 162 forks source link

Missing the concept of VST3_PATH environment #67

Open hfiguiere opened 4 years ago

hfiguiere commented 4 years ago

The spec is missing, for Linux, the concept of a VST3_PATH environment variable which would provide a list of search path to look for plugins.

Legacy VST (2) host on Linux support either VST_PATH or LXVST_PATH, and other plugin formats like LV2 of LADSPA have a similar concept.

VST3_PATH would provide a colon separated list of directory where the host would look up for plugins.

While the spec doesn't forbid it per see, some implementors are shy of doing it.

This would provide a great deal to help sandboxing audio applications and plugins.

hfiguiere commented 4 years ago

Just to clarify: these directory would be added to the search list and not replace standard ones.

x42 commented 4 years ago

Suggestion: Add the following

"Note: If needed, a host may use the $VST3_PATH environment variable to override the default search locations. Like the PATH variable for programs, it is colon-delimited on Unix, and searched from left to right."

to https://steinbergmedia.github.io/vst3_doc/vstinterfaces/vst3loc.html#linuxlocation


Some distributions do not have a writable /usr, but he main use-case are Linux application bundles like flatpak, appimage that sandbox applications and plugins. $APPFOLDER does not work there either since plugins are shared in their respective sandbox.

Some hosts (e.g. Carla) have already started to use VST3_PATH environment variable, and it would be preferable to standardize this before it becomes a too wide-spread ad-hoc extensions. -- I am somewhat reluctant to support this in Ardour and Mixbus without further discussion.

I do however disagree that it should be added to. All Unix environment *PATH replace the existing default (and no not add to).

A comparable good precedent is https://lv2plug.in/pages/filesystem-hierarchy-standard.html A standard path is defined and there LV2_PATH allows to override this

scheffle commented 4 years ago

Hi, there was a reason why the VST3 standard specifies the location where to find plug-ins. In the past the user had the chance to specify unlimited recursively scanned folders to the VST 2 search path. This led to a massive amount of support work for plug-in developers and host developers. So if the current specified locations don't work for some use cases on Linux, please describe the use cases and why the current specification does not work. Thank you.

x42 commented 4 years ago

As host developer I very much appreciate that the VST3 path is part of the spec and not customizable. Except we now get "but you can customize it in {Reaper|Live|Bitwig}" feedback :(

The specific case, @hfiguiere creates a https://en.wikipedia.org/wiki/Flatpak package of Ardour. The application and plugins are in a sandbox (here: self-contained file-system structure), for flapakt the plugins would reside in /app/extensions/Plugins/vst3.

We're trying to find a minimal consensus, for one-off special cases that are useful to packagers, while still preventing casual users from messing up things.


While on the subject:

https://steinbergmedia.github.io/vst3_doc/vstinterfaces/vst3loc.html mentions "The host recursively scans these folders at startup in this order" for Windows and Mac, but not for Linux. I assume this is an oversight and the same also applies to Linux hosts.

Also the bundle already has the architecture plugin.vst3/Contents/x86_64-linux/ plugin.vst3/Contents/i386-linux/ So explicitly allowing "/usr/lib32/vst3/ 32bit Plug-ins on 64bit OS" seems backwards.

hfiguiere commented 4 years ago

My specific case in Flatpak, but my request is just for a general solution to provide a standard way to specific supplementary lookup path for VST3 audio plugins.

One of the biggest constraints here is that in Flatpak /usr is read only and immutable. That mean the user installation in /usr or /usr/local prefix is not possible. And in that case the /usr isn't the same as the one on the operating system.

The application and any dependency not part of the runtime in installed in /app. This is also read-only at runtime, but the application packager can do whatever they want. /app/lib/vst3 could be considered, albeit it's not on the list and I doubt it is what you consider being $APPFOLDER/vst3 in the spec.

Flatpak support extension packages which are literally plugins that will be loaded when the application is loaded (their filesystem mounted in the right place). As for the audio plugins extensions, the choice has been made to have them in /app/extensions/Plugins and have for each type a sub directory in there. So VST3 plugins end up in /app/extensions/Plugins/vst3. And for each application that support audio plugins we set the environment in the Flatpak so there is a LADSPA_PATH, LV2_PATH, VST_PATH (sometime LXVST_PATH). This make the DAW find the plugins without user intervention, but it's up to the DAW to support it (most do).

For VST3, there is no specified mechanism (I can't say it has been for VST2 either as I don't have access to the spec, but it has worked so far).

There is one workaround that work, but I don't like because it require user configuration (or some other mechanism) is to put a symlink for /app/extension/Plugins/vst3 in $HOME/.vst3(it currently works with Ardour - but a stronger sandboxing could possibly prevent this) granted that the "recursively scan" mention missing for the Linux part of the spec is just an oversight.

Hence my request to be able to specify a custom path where to look for system wide installed VST3 plugins.

hfiguiere commented 4 years ago

Also the bundle already has the architecture plugin.vst3/Contents/x86_64-linux/ plugin.vst3/Contents/i386-linux/ So explicitly allowing "/usr/lib32/vst3/ 32bit Plug-ins on 64bit OS" seems backwards.

Also, /usr/lib32 doesn't exist on RedHat based Linux distro (it is mostly Debian based system like Ubuntu). On these, x86_64 (aka Intel 64-bits), /usr/lib is for 32-bit and /usr/lib64 is for 64-bits (native bit depth). So based on your spec, following the system convention would break one way on Intel 64-bits since /usr/lib is meant to have 32-bits. Since vst3 are bundles, I think having just lib and no lib32 would be ok.

ygrabit commented 4 years ago

Hi, what we mean with $APPFOLDER/vst3 is for sandboxing / local plugins to an application, only this application could see, load and use these plugins. This is the parallel to what we have on Mac : The VST3 folder is part of the application bundle, on Windows it is a subfolder to the folder where the exe is located. For example Cubase: "C:\Program Files\Steinberg\Cubase 8\VST3" where C:\Program Files\Steinberg\Cubase 8\cubase8.exe. Is that not enough ?

hfiguiere commented 4 years ago

No. It's like /Library/Audio/Plug-ins/VST3/ on macOS: system wide, but not in the OS filesystem. (and in the case inside the sandbox, but all the DAW will use the same set). Except that the path is specific to this setup.

This is not an alternative to the application provided VST which would be specific to the application.

scheffle commented 4 years ago

Hi, if the choice was made to have the plug-ins at /app/extensions/Plugins/vst3, why can this not be hardcoded into the apps as a search path for the plug-ins? If you build the flatpak of Ardour you sure need to build Ardour from source, don't you?

hfiguiere commented 4 years ago

This is not about Ardour. Ardour just happen to be one of the few VST3 host (still in the works) on Linux, and the one that triggered the conversation. In that case, /app/extensions/Plugins/vst3 is just for Flatpak packaging. Also I can not patch software I don't have the source code, while they can still be packaged as a flatpak, like BitwigStudio.

Having an environment variable part of the VST3 standard to specify an extra search path, and having application developers follow it go a very long way in offering the flexibility required for a more modern and secure approach to application distribution on Linux.

So I described why the spec doesn't work for me (us and them).

x42 commented 4 years ago

As for Ardour, I think I'll cut this short.

There are various other hosts that already allow to customize the path, so we'll likely also have to bite that bullet. Now the question is if we allow to pre-seed this path. e.g. At first start pick up $VST3_PATH if that varible is set and use that.

Right now I best of both worlds is likely: 1) always use the locations as specified by the standard 2) parse additional, optional path set by the user after those and pre-seed the user-path using $VST3_PATH if it's set at first app start. default is empty.

scheffle commented 4 years ago

Thanks for your additional input. Please try to understand us, we need to get a complete picture before we can change the spec. Can you point us to the specification of Flatpak where the plug-in mechanism is described? I could not find it.

hfiguiere commented 4 years ago

Again my request is not to be specific about Flatpak, but to have a generic solution that would fit, among others, Flatpak.

As or your question, I'll try to provide some links:

The basic concepts: https://docs.flatpak.org/en/latest/basic-concepts.html This gives an overview on Flatpak.

Extensions: https://github.com/flatpak/flatpak/wiki/Extensions This is from the point of view of the runtime.

To be fair I had to figure out a lot of things due to lack of documentation.

The part I wrote about how audio plugins are packaged: https://github.com/flathub/org.freedesktop.LinuxAudio.BaseExtension/blob/branch/20.08/README.md

If you have questions, I'd be happy to answer.

tobiasBora commented 1 year ago

Any news on this? I'm also in favor of VST3_PATH for a similar reason: NixOs does not provide /usr (to get a "pure" system), and I'll also be interested by VST3_PATH in the context of programs installed by users with no root access (with Nix you can package and install software without root access, configure environment variables automatically…)

JohnCorby commented 10 months ago

I'm also heavily in support of this. My main location is on a drive that has little space, so I usually only install VST2 plugins because they let me move them to a different drive with more space on it. I know VST3 is great but I really would like to use it in a place that doesn't fill up my computer so I have no more space on it :)

scheffle commented 10 months ago

@JohnCorby , you can already move vst3 plug-ins to another location and make a symlink to the original location.

JohnCorby commented 10 months ago

@scheffle of course. it would just be nice to not have to do that. especially on windows, because i do not trust windows.

PowerUser64 commented 9 months ago

The main problem is this makes it much harder to use vst3 in situations where plugins aren't stored in the same location because there's no standard way to add search paths.

ygrabit commented 9 months ago

We do not want to provide and standardize an env PATH for Linux. We had in the past with VST2 a lot of issue and misunderstanding for end users. What we could list in the VST SDK is per distribution/sandboxing/packaging a path like: Flatpak => /app/extension/Plugins/vst3 what do you think about this?

tobiasBora commented 9 months ago

I'm quite surprised, what kind of issues did you encounter? It might help to know them to better prevent them in the future.

Also, I don't see how this could be easier to maintain: each software would need to have a list of all per-distribution paths instead of to just read one environment variable? What if we realize that a packaging app was forgotten? Should we update the code of all softwares, meaning that in the meantime some softwares might read the /app/* folder while others would not? Realistically, this seems significantly more error prone than a simple environment variable.

Also, it would not solve the issue we have with Nix (I expect @PowerUser64 to also have this same issue). In Nix, we can install a software completely locally to a given shell, without using any sort of sandboxing. This is used for instance in this project https://codeberg.org/PowerUser/universal-studio (the author can certainly give more details), that allows people to run a full music environment in a shell, with many VST pluging (without installing anything on the computer!) with something as simple as:

$ nix run git+https://codeberg.org/PowerUser/universal-studio#ardour

This is really handy to be sure that many people writing on the same song use exactly the same set of plugins, allowing great reproducibility. This is made possible thanks to these lines https://codeberg.org/PowerUser/universal-studio/src/commit/676abb00d23b1a1c5d4db4785de1159f96a2d69c/flake.nix#L152-L158 unfortunately it will not work great for VST3 plugins if the app does not read VST3_PATH

In the worst case, if flatpak would benefit from reading something in /app/extension/Plugins/vst3, we could imagine both solutions, like read both /app/extension/Plugins/vst3 and VST3_PATH (to be honest, I would also be in favor of introducing VST3_PATH_ONLY, so that if VST3_PATH_ONLY is set, only plugin in this path are used, which is cool to be sure that people would only use VST distributed with universal-studio, again for better reproducibility… but let's start simple maybe)