search

sccn / liblsl

C++ lsl library for multi-modal time-synched data transmission over the local network
Other
108 stars 63 forks source link

New distribution channels #110

Open cboulay opened 3 years ago

cboulay commented 3 years ago

It would be great if liblsl could be installed simply with a one-liner on Mac and Linux platforms, and with an installer on Windows.

Debian / Ubuntu

The ultimate goal is to get liblsl into the official list of managed Debian packages. But that might take some time and requires mentorship and sponsorship.

In the interim, we can setup our own PPA that users can add, then they can install and update liblsl like any other package. What's the equivalent for raspbian?

More info:

Mac

We now have our own Tap: https://github.com/labstreaminglayer/homebrew-tap. This has a formula for liblsl and LabRecorder. We can add other formulae too. It would be nice if LabRecorder was a cask so it was added to /Applications, but that can come later.

Would we ever want to request to move the liblsl formula to homebrew/core? They've deprecated homebrew/science. They have brewsci/science but I'm not sure this is better for us than labstreaminglayer/tap.

Windows

The benefits to putting LSL into a package manager in Windows are small relative to the other platforms because the available package managers (in their simplest use-case) don't put libraries on the PATH. So this would only benefit developers. This isn't a bad goal, but liblsl is already pretty easy to get for any Windows developer so the gains here are small(er).

Discussion on NuGet package: https://github.com/labstreaminglayer/liblsl-Csharp/pull/17

I think we should work toward making an installer (.exe or .msi) and install to Program Files. Once that's in place then we can think about distribution via chocolatey or winget, both of which can be used as an installer delivery service.

Conda

There's already a conda-build file thanks to @tstenner. I think Tristan has a conda channel setup but we should probably make a labstreaminglayer org conda channel, then we can manage multiple applications there without hijacking Tristan's credentials.

Or even better would be to get it in conda-forge: https://github.com/conda-forge/staged-recipes/pull/15252

More info:

tstenner commented 3 years ago

Debian / Ubuntu

Setting up a private apt repository isn't for the faint of heart, but it's certainly possible. Launchpad makes it easy to host (Ubuntu only) PPAs; NeuroDebian might be easier to get into than the main debian distribution.

I think we should work toward making an installer (.exe or .msi) and install to Program Files.

That's easy. ninja install; cpack -G WIX.

There's already a conda-build file thanks to @tstenner. I think Tristan has a conda channel setup but we should probably make a labstreaminglayer org conda channel, then we can manage multiple applications there without hijacking Tristan's credentials.

Good idea. Maybe even split it up into "LSL stable", "LSL testing" and "LSL nightly"?

dmedine commented 3 years ago

Windows

I don't know that Program Files is a suitable destination for a dll. Qt, for example, uses its own directory structure in C: and this includes the dll files. There is another option, which is to actually have an installer that shoves liblsl onto the registry. This can cause incredible haywire for people using lots of different apps and wrappers using different builds, but will make things a lot smoother for users who do hygienic installations---i.e. all LSL dependent programs use the same copy of one dll (the way it works on sane operating systems based on Unix).

Due to historical reasons, this is a pretty dirty path for lsl on Windows so I'm not convinced it is the correct thing to do. Only a unified approach to app distribution as well as the C#, Matlab, and Python wrappers (does anyone use Java anymore?) would make this neat and tidy and I don't see that happening anytime soon.

I am also less optimistic than tstenner about the ease of generating installers using ninja and cpack. I haven't had much luck getting those tools to do what I want them to with WIX projects.

As unappetizing as it is, a .bat script can very easily put the dll file anywhere and add a line to the path. Right-click and run as admin is not too hard for average users to handle.

I think for Windows developers, any system wide install is probably just a debugging headache waiting to happen.

cboulay commented 3 years ago

@dmedine - Yes I agree. The whole thing is messy. Bunding lsl.dll in the App is still the preferred solution in Windows. But we're getting some complaints from developers who want to use liblsl as a dependency in other projects that there's no good way to do that, and we're also getting complaints that the .so's/.dylib's we provide don't work on specific OS versions, especially in pylsl. I'm sure this will only get worse with the Mac M1.

So we need proper system-level integration for Mac and Linux, and we'll provide a route for Windows too so the developers who want to include liblsl as a package-manager dependency don't have to abandon their Windows users. I'm sorry I neglected to provide an important detail in the initial post: Some of those Windows package managers (i.e., chocalety and winget) are just wrappers around windows installers. We don't actually need to release the windows installer publicly; we can build it then upload it somewhere the package managers can access.

Edit: As for the language bindings / wrappers, consistent with above I think the Windows versions should include the DLLs but Mac and Linux should rely on the system-level binary. For example, for pylsl this means having 2+ windows wheels and then a source-only release for every other platform (including raspbian). It kind of sucks that this means we aren't using the same general approach on all platforms.

tstenner commented 3 years ago

I am also less optimistic than tstenner about the ease of generating installers using ninja and cpack. I haven't had much luck getting those tools to do what I want them to with WIX projects.

My impression were mostly positive, even with multiple apps sharing Qt libraries. Still, Windows has the "every app for itself" model where everyone packages everything and makes sure nothing is shared with other libs and I'd see why developers would want that (on windows).

Only a unified approach to app distribution as well as the C#, Matlab, and Python wrappers (does anyone use Java anymore?) would make this neat and tidy and I don't see that happening anytime soon.

C# has nuget and Python has both pypi and conda. In theory, conda can even be used to build and distribute apps like LabRecorder.

As for the language bindings / wrappers, consistent with above I think the Windows versions should include the DLLs but Mac and Linux should rely on the system-level binary.

+1

It kind of sucks that this means we aren't using the same general approach on all platforms.

Well, we'd have the same approach on all modern Unix platforms, or rather all platforms with a sensible package manager.

dmedine commented 3 years ago

I just want to clarify that I am 100% in favor of cleaning up the distribution process! But given the cross-platform/cross-language nature of LSL it is quite challenging, even with all the automatic build/package tools currently available. I would propose making a list of distribution managers and app/lib combos to be targeted and prioritize them according to demand. Then we can divvy up the labor.

But we're getting some complaints from developers who want to use liblsl as a dependency in other projects that there's no good way to do that

I should check the slack channel more to understand this better, but I don't see this as an LSL problem but rather a Windows problem, and also a C/C++ problem. It is hard setting up any C++ development project on Windows that links to any 3rd party library---even the pro-shop ones like Qt that have plugins for Visual Studio. Part of being a good C/C++ developer is being able to deal with this issue. Sorry, but there is no free lunch.

Nuget is very good and we should distribute the C# wrapper there.

tstenner commented 3 years ago

Nuget is very good and we should distribute the C# wrapper there.

Still needs a native library, but other than that most of it is already done here

cboulay commented 3 years ago

Update - One can now install liblsl on MacOS with brew install labstreaminglayer/tap/lsl. I verified this works as a dependency for building LabRecorder. (Side note: after one uses the labstreaminglayer/tap prefix once, or otherwise does a brew tap labstreaminglayer/tap, that prefix is then no longer necessary)

If you're a Mac user and a LabRecorder user then I'd appreciate your feedback on this related issue: https://github.com/labstreaminglayer/App-LabRecorder/issues/65

tstenner commented 3 years ago

Update: nuget packages are discussed in https://github.com/labstreaminglayer/liblsl-Csharp/pull/17 and conda-forge packages in https://github.com/conda-forge/staged-recipes/pull/15252

tstenner commented 3 years ago

Update: https://anaconda.org/conda-forge/liblsl :tada:

tobiasherzke commented 2 years ago

Debian / Ubuntu

Setting up a private apt repository isn't for the faint of heart, but it's certainly possible. Launchpad makes it easy to host (Ubuntu only) PPAs; NeuroDebian might be easier to get into than the main debian distribution.

FWIW, we are distributing liblsl deb packages as part of our apt repository. For Debian's "amd64", "armhf" and "arm64" architecture. But this is restricted to the liblsl versions (currently 1.14) that we use, and the ubuntu releases / architectures that we use it on. We use "aptly" to manage and sign our apt repositories. For user documentation how to add our apt repo, see the beginning of file https://github.com/HoerTech-gGmbH/openMHA/blob/master/INSTALLATION.md. If you want to start your own apt repository, that would be nice, and we would probably stop distributing liblsl ourselves via apt.

There is one versioning issue with your deb packages that I will bring to your attention in a separate github issue, #140. You may want to solve that before starting your own apt repository.