Tracking ossia-score in the package ecosystem

luzpaz commented 1 year ago

Note: the above badge will auto-update when OS packaging repos and distros incorporate the ossia-score package.

Other

[ ] Wikipedia page (so we can also have a wikidata page which will be displayed on ossia-score repology badge example: Ardour (wikipedia page) Ardour (wikidata page) Ardour wikidata badge

jcelerier commented 1 year ago

well, I think it hasn't been packaged for any other distro aha

luzpaz commented 1 year ago

Got it. Closing.

jcelerier commented 1 year ago

if anyone can help for doing it it would be great though!

jcelerier commented 1 year ago

maybe we can use this issue to plan how to get score in more distros?

there's also https://github.com/NixOS/nixpkgs/pull/174802 for getting it into Nix

I think the best would be to try toget a debian package

luzpaz commented 1 year ago

I think the best would be to try toget a debian package

Agreed. And by the same logic: getting an Arch package (which will permeate to manjaro and other arch clones)

Edit: updated the OP

jcelerier commented 1 year ago

@dvzrv do you know what would be the steps to take to migrate https://aur.archlinux.org/packages/ossia-score from AUR to community? thanks!

jcelerier commented 1 year ago

working on the mingw packages... it already builds entirely against it on the CI so shouldn't be too hard (likewise arch is checked in the CI)

jcelerier commented 1 year ago

For debian the simplest looks like it'd be by using https://gitlab.kitware.com/debian/dh-cmake

jcelerier commented 1 year ago

@yurivict do you know what we should do to get ossia in FreeBSD ports? Ideally I'd like to set-up some form of CI to make sure we don't break it but I honestly have no idea on how to do it on GH Actions...

dvzrv commented 1 year ago

@dvzrv do you know what would be the steps to take to migrate https://aur.archlinux.org/packages/ossia-score from AUR to community? thanks!

Generally speaking, I'd be interested in maintaining score and libossia for Arch Linux. However, I see major road blocks for both projects in their current form:

There are many bundled libraries in score and libossia, most of which are readily available in downstream distributions
everything is statically linked (which is only used under very rare circumstances on Arch)
vendored libs are in most cases not upstream directly
the cmake setup in score and libossia does not allow to devendor any of these libs

Judging from the above I do not believe I have the time or energy to keep up with a devendoring effort of this scale.

Moving forward, I would reconsider if the build setup would allow devendoring libraries that do not have substantial modifications done by you to make the ossia projects work and which are available (or can be made available) on downstream distributions. Libraries that are not necessary for the Linux target should not be pulled in at all. Libraries that are not headers-only should be dynamically linked. Generally, I would appreciate if you could target latest stable releases of libraries (this is also useful to keep in sync with pinned versions via submodules for your own build purposes) and no pre-release versions (I can recommend nvchecker for the purpose of tracking upstream releases). This may require contacting upstreams to create releases.

libossia:

[x] { in Arch: catch2 / catch2v3 } Catch2 => only relevant for the development of score, not for releases
[ ] { header-only, not in Arch } Flicks => only used for showing its LICENSE in the About dialog
[ ] { header-only, not in Arch } PerlinNoise
[ ] { header-only, not in Arch, } Servus => heavily modified: https://github.com/HBPVIS/Servus/compare/master...jcelerier:Servus:master
[ ] { header-only, not in Arch } SmallFunction
[x] { header-only, in AUR:compile-time-regular-expressions } compile-time-regular-expressions => version 3.3.2 in AUR does not build, it needs latest released version 3.7
[ ] { in AUR:concurrentqueue } concurrentqueue
[ ] { header-only, not in Arch } dno => @thibaudk is the maintainer, I don't know if this is used anywhere outside ossia
[ ] { header-only, not in Arch } dr_libs
[x] { header-only, in AUR } exprtk
[x] { in Arch, but not up-to-date } fmt => version 10 released in May is required, Arch only has version 9
[ ] { irrelevant to score } ios-cmake
[x] { header-only, in AUR } kfr => does not seem to export the symbols for the DFT
[ ] { not in Arch } libartnet
[ ] { in AUR: libe131-git } libe131=> only used for the LICENSE as only one function was taken and modified, i
[ ] { not in Arch } libremidi => I'm the maintainer. It can be used header-only or not.
[x] libsamplerate
[ ] { irrelevant to score } max-sdk
[ ] { header-only, not in Arch } mdspan
[ ] { header-only, not in Arch } nano-signal-slot
[ ] { header-only, in Arch: oscpack } oscpack => pretty much entirely rewritten for C++11 support compared to the original: https://github.com/svn2github/oscpack/compare/master...jcelerier:oscpack:master
[ ] { irrelevant to score } pybind11
[ ] { not in Arch } r8brain-free-src
[x] { in Arch: rapidfuzz-cpp } rapidfuzz-cpp
[x] { header-only, in Arch: rapidjson } rapidjson (I see compiler warnings&errors when building with it that indicates that Arch ships a PRETTY OLD, fairly insecure & buggy version of it... last official release is from 2016 but the library still gets updates in master branch ; e.g. the Arch version does not even have https://github.com/Tencent/rapidjson/commit/5cd62c235d0ff1ccc2f9822047a470f62ad947f6 which is 7 years old)
[x] { in Arch: re2 } re2
[ ] { header-only, slightly patched, in AUR: readerwriterqueue-git } readerwriterqueue
[ ] { not in Arch, I'm the maintainer }: rnd
[x] { in Arch: rubberband } rubberband. This one has been modified a bit but mainly for CMake support... will check if there are some other deep things.
[ ] { header-only, in AUR: span-git } span
[x] { in Arch } spdlog
[ ] { not in Arch } timertt
[ ] { header-only, not in Arch } tuplet
[ ] { header-only, not in Arch } unordered_dense
[ ] { header-only, in AUR: verdigris-git } verdigris -> my version is modified a little bit
[ ] { header-only, not in Arch } weakjack
[ ] { header-only, in Arch: websocketpp } websocketpp -> i'm in develop branch as master branch / current release 0.8.2 (from 2020) does not build in C++20 mode
[ ] { not in Arch } whereami (there is another "whereami" library in Arch which is unrelated to this)
[x] { in Arch } wiiuse. we have some patches but they are not relevant to running in Arch, it's more for enabling the library to be used when bluez is not installed but the maintainers did not want to merge them

score:

[ ] { in AUR: aether.lv2 } Aether -> this one is heavily modified to be integrated as a score plug-in: https://github.com/Dougal-s/Aether/compare/master...jcelerier:Aether:master
[ ] { not in Arch } DSPFilters
[ ] { not in Arch } Gamma
[ ] { not in Arch } Gist -> kinda modified (https://github.com/adamstark/Gist/compare/master...jcelerier:Gist:master)
[ ] { in Arch but... } Guitarix -> this one is not actually Guitarix, these are data files used by some Faust presets.. these tables aren't exported by guitarix so I had to pry them out of the codebase and turn it into a library
[ ] { not in Arch } QCodeEditor -> heavily modified, upstream does not maintain it : https://github.com/jcelerier/qcodeeditor
[ ] { not in Arch } QMenuView -> heavily modified (ported to verdigris instead of Qt's moc)
[ ] { not in Arch } QProgressIndicator -> heavily modified (ported to verdigris instead of Qt's moc)
[ ] { not in Arch } Qt-Color-Widgets -> heavily modified (ported to verdigris instead of Qt's moc)
[ ] { in Arch but... } STK -> exact same situation than guitarix, just used for data tables for Faust
[ ] { irrelevant to Linux } Spout
[ ] { irrelevant to Linux } Syphon-Framework
[ ] { header-only, not in Arch } avendish -> i'm the maintainer.. and it advances really in lock-step with score like libossia
[ ] { not in Arch } hap
[ ] { ... } libossia
[ ] { not in Arch } libpd -> I see that Pd ships z_libpd.h but to work with score it has to be built with -DPDINSTANCES=1
[x] { in Arch } libsndfile
[ ] { not in Arch } llfio -> not too critical if it's not there, it just uses std::filesystem instead (which is a fair deal slower)
[ ] { not in Arch } magicitems -> only relevant for the development of score, not for releases
[x] { in Arch } mimalloc
[ ] { not in Arch } outcome -> only used as a dependency for LLFIO
[ ] { not in Arch } phantomstyle -> patched for Qt 6
[ ] { not in Arch } quickcpplib -> only used as a dependency for LLFIO
[ ] { not in Arch } shmdata
[x] { in Arch } snappy
[ ] { see comment below } vst3
[ ] { not in Arch } zipdownloader -> i'm the maintainer, it's just an internal score function that I turned into a library because I was using it in a few projects

Some of the above are a bit special as they are monstrous sdk targets (e.g. vst3sdk). Those I have by now dealt with in the context of providing them as source targets (e.g. below /usr/src/vst3sdk/), so that projects may rely on a custom variable to find those sources (e.g. PROJECT_VST3SDK_SOURCES=/usr/src/vst3sdk/).

jcelerier commented 1 year ago

thanks for your thorough answer @dvzrv ! do you mind if I check & edit your post for annotating the libraries with their state?

dvzrv commented 1 year ago

thanks for your thorough answer @dvzrv ! do you mind if I check & edit your post for annotating the libraries with their state?

Sure, please do! I mainly just added the 3rdparty subdirectories of both projects. Although some common libs I recognized right away, I didn't go into deeper investigation for them all (e.g. what version you're using them in vs. what is current latest or whether they are required on Linux, etc.)

jcelerier commented 1 year ago

okay, I did a round of them, here are the ones that I think can be devendored without too much difficulty:

Catch2
libsamplerate
compile-time-regular-expressions
concurrentqueue
exprtk
fmt
spdlog
rapidjson
re2
rubberband
wiiuse
libsndfile
mimalloc
snappy
vst3

which others would be critical for you?

jcelerier commented 1 year ago

by the way, I don't think it's too necessary to have a separate package for libossia, it is really built with entirely different & ABI-incompatible options and features when built for usage by score, than when built as a standalone library (where it is more used as network communication middleware for Max/MSP externals)

dvzrv commented 1 year ago

which others would be critical for you?

Thanks for going over the list! Much appreciated! I think even the header only libs can be packaged (with some we do already). However, some upstreams are a bit undead and don't necessarily even have releases.

I'd need to spend some time to figure out which are easy to package, so thanks for highlighting those that are not actually needed (or only in dev context). I'm currently on vacation so it might take me up to mid July to really get going on this.

jcelerier commented 1 year ago

okay, i added a -DSCORE_USE_SYSTEM_LIBRARIES=1 flag that will try to lookup system libs first. Will mark the ones for which it is done.

I encountered an issue with fmt for which we use v10 (the latest stable release from May) - score does not build with v9.

Note that as far as I can see all the external libs are linked dynamically.. do you have some specific ones in mind when you mention "everything is dynamically linked" ?

in any case thanks and enjoy your vacations!

yurivict commented 1 year ago

do you know what we should do to get ossia in FreeBSD ports?

Last time a massive amount of bundled dependencies prevented porting of ossia to FreeBSD. Some bundled dependencies probably conflicted with pre-installed packages. Now there's still a massive amount of bundled dependencies present in the project.

Having all dependencies bundled with the project is just not a sustainable practice, because if all projects would do this they would require exponentially long time to build, because every package would be rebuilt many times for many dependencies.

jcelerier commented 1 year ago

@yurivict from the list I put of the libraries "not in Arch", do you see some that would be present in freebsd so that I can try detecting them?

yurivict commented 1 year ago

@jcelerier where is this list? The string "not in Arch" isn't in the source tree, and configure doesn't print it.

jcelerier commented 1 year ago

@yurivict i'm referring to the post a bit above this one: https://github.com/ossia/score/issues/1476#issuecomment-1612618879

yurivict commented 1 year ago

@jcelerier We now have PerlinNoise, libartnet, libremidi, mdspan, nano-signal-slot ports in FreeBSD.

jcelerier commented 1 year ago

last master commits are going to look for them (and all others) through cmake find_package.. installing a VM to try

luzpaz commented 1 year ago

Amazing packaging work happening here. Props to all involved!

jcelerier commented 1 year ago

@ycollet this issue may be relevant... do you know what would be the steps to get into Fedora proper? cheers!

ycollet commented 1 year ago

I still have it in the audinux repository: https://copr.fedorainfracloud.org/coprs/ycollet/audinux/

ycollet commented 1 year ago

But for the main Fedora repository, a fedora packager must adopt the spec and manage to put it in the fedora repository. I am not a fedora packager ... So, for now, I still build ossia in audinux :)

luzpaz commented 1 year ago

Opened FreeBSD new port request: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=272606

yurivict commented 1 year ago

I have a question: can the dependency llfio be made optional? It doesn't easily compile on FreeBSD: many C++ errors in the code.

jcelerier commented 1 year ago

@yurivict yes it is optional, if it is not part of the build it justy won't be used

jcelerier commented 1 year ago

(it'd be nice to get it to work on freebsd though... on my linux it iterates so much faster compared to std::filesystem or fts it's not even funny, on folders with a few hundred thousand files... and I don't even use it in a particularly optimized way). for the record it's only used from this file: https://github.com/ossia/score/blob/1ba227378059faa674f2b623bf6db41db14cd902/src/lib/score/tools/RecursiveWatch.cpp

luzpaz commented 1 year ago

Note: I opened a repology report asking to merge 'score' packages in to 'ossia-score' (https://repology.org/project/score/report)

Edit: repology has merged packages now under ossia-score. :+1:

yurivict commented 1 year ago

How to disable pre-compiled headers?

I am getting this gcc-12 failure on FreeBSD:

grep -r "sorry, unimplemented: PCH allocation failure

There is the SCORE_PCH option which is OFF, but pre-compiled headers are still used.

The message "PCH allocation failure" appears in the gcc source.

GCC bug report

yurivict commented 1 year ago

This is a showstopper for the FreeBSD port. Anybody knows how to solve this?

jcelerier commented 1 year ago

I'll be looking at it asap next week, kinda busy with another project this week

luzpaz commented 1 year ago

Update:

winget package added
chocolatey package request made
gentoo package request made

RoozbehOssia commented 1 year ago

🌈🌞🍀🫶🏽

On Sun 23. Jul 2023 at 19:33, luzpaz @.***> wrote:

Update:

winget package added

chocolatey package request made

— Reply to this email directly, view it on GitHub https://github.com/ossia/score/issues/1476#issuecomment-1646899616, or unsubscribe https://github.com/notifications/unsubscribe-auth/AENTXP7A4FUXOAA2E73CUQTXRVN7ZANCNFSM6AAAAAAZSDRPKA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

-- null

luzpaz commented 1 year ago

Chocolatey package running into issues (old issues apparently: https://community.chocolatey.org/packages/ossia-score). Since it was rejected back then, it's rejected now (https://github.com/chocolatey-community/chocolatey-package-requests/issues/1435#issuecomment-1665384923)

jcelerier commented 1 year ago

3.1.12 released which should improve the packaging story, it has the work done this summer

luzpaz commented 1 year ago

bumped winget and freschcode.club repos. Those should update soon.

luzpaz commented 11 months ago

bumped some downstream tickets so we inch closer toward the goal of this ticket

jcelerier commented 9 months ago

3.1.13 released :)

luzpaz commented 9 months ago

Bumped existing downstream repos. Badge should reflect that soon.

jcelerier commented 6 months ago

3.2.0 released. added repology to the readme

jcelerier commented 4 months ago

NixOS merged it :) also there's a flatpak ready to be merged : https://github.com/flathub/flathub/pull/5376 @yurivict @dvzrv @ycollet anything I can do to help packaging on upstream distros ?

jcelerier commented 4 months ago

@yurivict btw I added a FreeBSD CI step which is finally green : https://github.com/ossia/score/actions/runs/9827416259/job/27130071142

jcelerier commented 4 months ago

flatpak available :) https://flathub.org/fr/apps/io.ossia.score

ycollet commented 4 months ago

On my side, I will work on the package probably in august.

jcelerier commented 4 months ago

great, please tell me if it's possible to help in any way !

dvzrv commented 4 months ago

Same here. I'm very busy this month and likely into half of the next.

If you could provide more detailed information on the optional dependencies, that'd be great. In https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=ossia-score#n11 you provide many, but they are all unnamed. Does the application link against some or all of them? Is any of that dlopen'ed?

jcelerier commented 4 months ago

'pipewire'

dlopen'd

'faust'

linked directly, if the package is built against faust it will require it at runtime. (but faust is not necessary for ossia to work, it's just additional albeit useful features). Though I could change it to being dlopen'd without much issues, there's only 3/4 function calls required.

'lilv' 'lv2'

linked directly so if ossia is built with lv2 support they are required at runtime (and it's way too much work to dlopen Lilv)

'suil'

dlopen'd. Note that for LV2 UI support, this requires this patch as upstream suil does not support Qt 6 yet : https://github.com/lv2/suil/pull/21

'sdl2'

used for gamepad support, linked directly

'libfreenect2'

used for kinect support, linked directly. very, very, very optional.

'ysfx'

used for JSFX plug-ins support, linked directly

'ultraleap-hand-tracking-service'

used for leapmotion / ultrealeap support, dlopen'd

'gpsd'

used for GPSd support, dlopen'd

'bluez'

used for Bluetooth / BLE support, accessed through D-Bus, it should work without it available (but I am not able to remove it without removing half of my system to make sure to be honest... I do a run-time check nevertheless)

'ndi-sdk'

used for NDI support, dlopen'd

'libcoap'

used for CoAP protocol support, linked directly

'avahi'

used in a lot of places for DNS-SD / Bonjour support. dlopen'd though the experience may be really subpar without it.

ossia / score

Tracking ossia-score in the package ecosystem #1476

Linux

FreeBSD

macOS

Windows

Other