search

gogins / csound-extended

Extensions for Csound including algorithmic composition, Android app, and WebAssembly.
GNU Lesser General Public License v2.1
40 stars 1 forks source link

Create Linux package for csound-extended #50

Closed gogins closed 5 years ago

gogins commented 6 years ago

It appears that the firm policy of Victor, Steven, and the other core Csound developers now is that the core csound repository be simplified, and accordingly that plugin opcodes and other such extensions be maintained wherever possible in separate repositories.

Personally, I think this is great for the developers but not so great for actual Csound users. But I'm not going to fight it, that would be a waste of breath.

I have now mentally adjusted to this new situation. The goal here is to simplify the maintenance of my contributions to computer music software, which in turn is strictly secondary to my primary goal of composing music. Here is my policy:

  1. It must be possible to clone this repository on Ubuntu, and build everything in the standard way.
  2. All of my extensions are to be maintained in this repository, with the exception of the VST features which will not be maintained here, and released only as binary-only freeware.
  3. At this time, I am going to maintain only Ubuntu releases. Hopefully they will work on any Debian distribution.
  4. All releases are to be 64 bit only, though I will attempt to maintain 32 bit compatibility.
  5. For release format, I will of course provide source packages. In addition, I may provide a PPA or try to get into Debian. For now, I will do only source packages and binary archives.
  6. I will simplify the build system as follows: 1 Minimize submodules, use installed packages wherever possible. 2 Get rid of all scripts, use only CMake and defined targets. 3 Look into current best practices for complicated CMake projects. 4 Look into using FindCsound.cmake. If it works for /usr/local, I may break this repository up into parts.
gogins commented 6 years ago

I think the pattern is called "superbuild" and involves a top-level CMakeLists.txt that uses only ExternalProject_add.

See also this: http://www.steveire.com/embracing-modern-cmake.pdf, though it doesn't describe a superbuild.

This looks good: https://www.slideshare.net/DanielPfeifer1/effective-cmake

http://simmesimme.github.io/lessons/2011/11/02/ppa-launchpad-cmake

gogins commented 6 years ago

I have renamed this issue.

The objective has been clarified: create a source distribution and PPA for csound-extended.

gogins commented 6 years ago

Re-doing to-dos:

  1. Remove all possible dependencies.
  2. Make as many as possible of the remaining dependencies be package dependencies, including Csound itself.
  3. Change to the CMake superbuild pattern.
  4. Put all tasks into CMake.
  5. Ensure that patches actually work.
  6. Create a working source distribution.
  7. Create a PPA.
  8. Create packages.
gogins commented 6 years ago

Unfortunately the last source package for Csound in Ubuntu is 6.05.

gogins commented 6 years ago

I have created a Launchpad account for myself and a PPA for csound-extended at https://launchpad.net/~gogins/+archive/ubuntu/csound-extended. I am now attempting to create the source package. Here is a guide: http://packaging.ubuntu.com/html/packaging-new-software.html

gogins commented 6 years ago

Initial version will be 0.2.0.

Question: Is it possible to have more than one package in the same PPA?

Answer: I think so, see https://askubuntu.com/questions/17508/how-to-have-debian-packaging-generate-two-packages-given-an-upstream-source-arch

gogins commented 6 years ago

Yikes! https://wiki.debian.org/PackagingWithGit#Packaging_with_Git

gogins commented 5 years ago

Possibly a snap is a better idea. See https://snapcraft.io/.

gogins commented 5 years ago

I have decided to pursue creating a snap unless that proves infeasible.

gogins commented 5 years ago

Whether or not I do a snap or a PPA I need a CMake superbuild. See:

gogins commented 5 years ago

I'm reorganizing. I got:

fatal: Not a git repository: modules/asio-standalone/../../../../../.git/modules/CsoundForAndroid/ndk-libraries/link/modules/modules/asio-standalone
fatal: 'git status --porcelain=2' failed in submodule src/CsoundForAndroid/ndk-libraries/link
gogins commented 5 years ago

I am looking at vcpkg, which apparently now works on Linux. I'm looking at the package list.

There is also buckaroo and conan.io.

ras0219-msft commented 5 years ago

Please feel free to @ me if you have any questions about vcpkg!

gogins commented 5 years ago

vcpkg does not have ports for several basic dependencies of extended-csound.

It's not currently feasible to install system packages from CMake itself.

It may be that to use snapcraft, I will have to install all system dependencies locally.

ras0219-msft commented 5 years ago

vcpkg does not have ports for several basic dependencies of extended-csound

Which dependencies, specifically?

gogins commented 5 years ago

Thanks for your response.

What I am trying to package is quite complex. First, there are three target systems, namely Ubuntu, WebAssembly, and Android. Also, I embed LuaJIT and Common Lisp. Also, I provide interfaces to Common Lisp, Lua and LuaJIT, Python, and Java.

I have created Debian packages locally to identify system dependencies:

Debian package contents:
Package: csound-extended
Version: 1.0.0
Section: devel
Priority: optional
Architecture: amd64
Depends: ecl, libasound2 (>= 1.0.16), libc6 (>= 2.17), libcurl3-gnutls (>= 7.16.2), libfltk-images1.3, libfltk1.3 (>= 1.3.1), libfluidsynth1 (>= 1.1.6-4~), libgcc1 (>= 1:4.0), libjack-jackd2-0 (>= 1.9.10+20150825) | libjack-0.125, libncurses5 (>= 6), libpng12-0 (>= 1.2.13-4), libportmidi0, libpulse0 (>= 0.99.1), libpython2.7 (>= 2.7), libqt5core5a (>= 5.9.0~beta), libqt5gui5 (>= 5.0.2), libqt5webchannel5 (>= 5.6.1), libqt5webenginewidgets5 (>= 5.7.1), libqt5widgets5 (>= 5.0.2), libsndfile1 (>= 1.0.20), libssl1.0.0 (>= 1.0.0), libstdc++6 (>= 6), libtinfo5 (>= 6), libx11-6, zlib1g (>= 1:1.1.4)

Then there are dependencies that I build from sources in my CMake build system:

link (header file only library, but I build a test program to test it)
cm2 (Common Lisp library)
csound
Eigen (header file only library)
gmm++ (header file only library)
fomus (Common Lisp library)
liblo
libmusicxml
libsndfile (also a system package; I need sources to build for Android)
luajit-2.0 (also a aystem package; I need sources to build for Android)
oboe
portaudio (I need to configure the build)
stk (also a system package; I need sources to build for Android)
gogins commented 5 years ago

This is looking fiddly. I may stop right here and go back to the idea of a PPA. For that, I need to pay attention to the Lintian errors and warnings.

gogins commented 5 years ago

This has turned into another one of those time sinks that I can no longer afford. I will attempt to tame it thus:

gogins commented 5 years ago 
Running lintian...
W: csound-extended: package-name-doesnt-match-sonames libCsoundAC libcsnd6-6.0 libcsound64-6.0 liblo7 libmusicxml2-3.00 libportaudio
E: csound-extended: unstripped-binary-or-object usr/bin/CsoundHtml5
E: csound-extended: unstripped-binary-or-object usr/bin/csound.node
E: csound-extended: changelog-file-missing-in-native-package
E: csound-extended: extended-description-is-empty
E: csound-extended: dir-or-file-in-home home/mkg/
E: csound-extended: dir-or-file-in-home home/mkg/.local/
E: csound-extended: dir-or-file-in-home home/mkg/.local/lib/
E: csound-extended: dir-or-file-in-home home/mkg/.local/lib/python2.7/
E: csound-extended: dir-or-file-in-home home/mkg/.local/lib/python2.7/site-packages/
E: csound-extended: dir-or-file-in-home home/mkg/.local/lib/python2.7/site-packages/CsoundAC.py
E: csound-extended: dir-or-file-in-home home/mkg/.local/lib/python2.7/site-packages/_CsoundAC.so
E: csound-extended: dir-or-file-in-home home/mkg/.local/lib/python2.7/site-packages/_csnd6.so
E: csound-extended: dir-or-file-in-home home/mkg/.local/lib/python2.7/site-packages/csnd6.py
E: csound-extended: dir-or-file-in-home home/mkg/.local/lib/python2.7/site-packages/ctcsound.py
E: csound-extended: dir-in-usr-local usr/local/share/
E: csound-extended: dir-in-usr-local usr/local/share/cmake/
E: csound-extended: dir-in-usr-local usr/local/share/cmake/Csound/
E: csound-extended: file-in-usr-local usr/local/share/cmake/Csound/FindCsound.cmake
W: csound-extended: file-in-unusual-dir usr/local/share/cmake/Csound/FindCsound.cmake
W: csound-extended: embedded-javascript-library usr/share/doc/csound-extended/html/jquery.js please use libjs-jquery
W: csound-extended: jar-not-in-usr-share usr/lib/csnd6.jar
W: csound-extended: binary-without-manpage usr/bin/CsoundHtml5
W: csound-extended: binary-without-manpage usr/bin/atsa
W: csound-extended: binary-without-manpage usr/bin/cs
W: csound-extended: binary-without-manpage usr/bin/csanalyze
W: csound-extended: binary-without-manpage usr/bin/csb64enc
W: csound-extended: binary-without-manpage usr/bin/csbeats
W: csound-extended: binary-without-manpage usr/bin/csdebugger
W: csound-extended: binary-without-manpage usr/bin/csound
W: csound-extended: binary-without-manpage usr/bin/csound.node
W: csound-extended: binary-without-manpage usr/bin/cvanal
W: csound-extended: binary-without-manpage usr/bin/dnoise
W: csound-extended: binary-without-manpage usr/bin/envext
W: csound-extended: binary-without-manpage usr/bin/extract
W: csound-extended: binary-without-manpage usr/bin/extractor
W: csound-extended: binary-without-manpage usr/bin/het_export
W: csound-extended: binary-without-manpage usr/bin/het_import
W: csound-extended: binary-without-manpage usr/bin/hetro
W: csound-extended: binary-without-manpage usr/bin/lpanal
W: csound-extended: binary-without-manpage usr/bin/lpc_export
W: csound-extended: binary-without-manpage usr/bin/lpc_import
W: csound-extended: binary-without-manpage usr/bin/makecsd
W: csound-extended: binary-without-manpage usr/bin/mixer
W: csound-extended: binary-without-manpage usr/bin/oscdump
W: csound-extended: binary-without-manpage usr/bin/oscsend
W: csound-extended: binary-without-manpage usr/bin/pv_export
W: csound-extended: binary-without-manpage usr/bin/pv_import
W: csound-extended: binary-without-manpage usr/bin/pvanal
W: csound-extended: binary-without-manpage usr/bin/pvlook
W: csound-extended: binary-without-manpage usr/bin/scale
W: csound-extended: binary-without-manpage usr/bin/scot
W: csound-extended: binary-without-manpage usr/bin/scsort
W: csound-extended: binary-without-manpage usr/bin/sdif2ad
W: csound-extended: binary-without-manpage usr/bin/sndinfo
W: csound-extended: binary-without-manpage usr/bin/srconv
W: csound-extended: binary-without-manpage usr/bin/xml2guido
W: csound-extended: binary-without-manpage usr/bin/xmltranspose
E: csound-extended: shlib-with-executable-bit usr/bin/csound.node 0755
E: csound-extended: sharedobject-in-library-directory-missing-soname usr/lib/lib_jcsound6.so
E: csound-extended: sharedobject-in-library-directory-missing-soname usr/lib/luaCsoundAC.so
W: csound-extended: non-dev-pkg-with-shlib-symlink usr/lib/libmusicxml2.so.3.00 usr/lib/libmusicxml2.so
W: csound-extended: non-dev-pkg-with-shlib-symlink usr/lib/liblo.so.7.3.0 usr/lib/liblo.so
W: csound-extended: non-dev-pkg-with-shlib-symlink usr/lib/libcsnd6.so.6.0 usr/lib/libcsnd6.so
W: csound-extended: non-dev-pkg-with-shlib-symlink usr/lib/libcsound64.so.6.0 usr/lib/libcsound64.so
W: csound-extended: non-dev-pkg-with-shlib-symlink usr/lib/libCsoundAC.so. usr/lib/libCsoundAC.so
W: csound-extended: shlib-without-versioned-soname usr/lib/libportaudio.so libportaudio.so
E: csound-extended: no-shlibs-control-file usr/lib/libmusicxml2.so.3.00
E: csound-extended: no-shlibs-control-file usr/lib/liblo.so.7.3.0
E: csound-extended: no-shlibs-control-file usr/lib/libcsnd6.so.6.0
E: csound-extended: no-shlibs-control-file usr/lib/libcsound64.so.6.0
E: csound-extended: no-shlibs-control-file usr/lib/libCsoundAC.so.
W: csound-extended: maintscript-calls-ldconfig postinst
E: csound-extended: package-must-activate-ldconfig-trigger usr/lib/libportaudio.so
W: csound-extended: maintscript-calls-ldconfig postrm
gogins commented 5 years ago

See https://salsa.debian.org/multimedia-team/csound/tree/master/debian.

gogins commented 5 years ago

The Debian and Ubuntu packages for Csound are more up to date (6.10 or 6.11) than I recall from last time I looked. I am considering removing the csound submodule and its dependencies from csound-extended, this would really simplify my life.

However on my main computer, my notebook, the OS is 17.10 (Artful Aardvark) and Csound installed as 6.09. This is not recent enough. Csound 6.11 is available in 18.10 (Cosmic Cuttlefish).

I could also produce my own Ubuntu package from the latest Debian source package.

I will see if I can upgrade my notebook to 18.10, if so, I will use the Csound system packages for csound-extended.

gogins commented 5 years ago

Cosmic Cuttlefish doesn't seem to be ready just yet. I will go with Ubuntu 18.04 (Bionic Beaver) and Csound 6.10.

I didn't think this through all the way. I still need the Csound sources to build my WebAssembly release and also for the Csound for Android app. So I have to leave the structure in place. But of course I should build the Ubuntu packages against the Ubuntu csound packages. Perhaps I should split up the repository.

  1. csound-extended
  2. csound-android
  3. csound-wasm
gogins commented 5 years ago

I am not splitting up the repository. I will just keep my different built scripts. I will change the csound-extended CMake build to use system packages for all parts of Csound.

gogins commented 5 years ago

There will be a package conflict with the original CsoundAC. This is manageable. My package will conflict with or replace the original package.

The Python things on my system go into /usr/lib/python2.7/dist-packages/_CsoundAC.x86_64-linux-gnu.so. This is hardcoded in the Debian control scripts.

gogins commented 5 years ago

This is going well and I am confident it will be usable in a day or so.

Useful example: https://code.cor-lab.org/svn/cca/trunk/cca/cpack/CPackDebianConfig.cmake https://github.com/FernetMenta/kodi-agile/blob/master/cmake/cpack/CPackConfigDEB.cmake

gogins commented 5 years ago

Everything seems to be building, except the Ubuntu software installer shows a proprietary license, which is not the case; on the other hand, other .deb packages also show a proprietary license when that is not the case. There also is a problem with unstripped binaries. I may need Lintian overrides for those.

gogins commented 5 years ago

Installed after removing the existing CsoundAC package. These are the wrong permissions:

-rwxr-xr-x 1 root root 368720 Oct 24 14:21 libcmask_opcodes.so
gogins commented 5 years ago

Update the following and ensure they install:

gogins commented 5 years ago

Before release run these tests:

This will require gathering or creating these tests, which should also be installed. Except as mentioned, tests will be in the test-examples directory. (Installation done, but needs a little bit of cleanup.)

gogins commented 5 years ago

Configure control to conflict with or replace the following:

/usr/lib/libCsoundAC.so libcsoundac-dev
/usr/lib/libCsoundAC.so.6.0 libcsoundac6.0v5
/usr/lib/lua/5.1/luaCsoundAC.so liblua5.1-luacsnd

Done.

gogins commented 5 years ago

I'm not picking up CppSound in the Lua interface.

gogins commented 5 years ago

It seems the system package for Csound is linked with the Lua libraries, not the LuaJIT libraries. This is a bug.

I will try the workaround of uninstalling Lua and installing LuaJIT on my system.

No help, that would also remove: csladspa csound csound-extended-dev csound-utils csound-vst csoundqt geeqie libcsnd-dev libcsnd6-6.0v5 libcsnd6-java libcsound64-6.0 libcsound64-dev liblua5.1-0 liblua5.1-0-dev python-csound.

There is a workaround on my system, to make a symbolic link from liblua5.1.so.0 to libluajit5.1.so.

The Android build explicitly links with the static luajit library.

gogins commented 5 years ago

This is a real mess because the liblua5.1-luacsnd package installs CsoundAC which it probably should not do. That means I can't just make liblua5.1-luacsnd a dependency of csound-extended.

gogins commented 5 years ago

It looks like I probably have to change CsoundAC to use csound_threaded.hpp instead of CppSound.hpp because cs_glue.hpp is not installed.

No, I need to expose all the methods of CppSound that are actually used in my examples as methods of MusicModel. I have done this (I think, more can be added).

gogins commented 5 years ago

Now working on the problem with the LuaJIT opcodes. I may have to drop the LuaJIT opcodes from the Android app.

See https://android.googlesource.com/platform/ndk/+/master/docs/ClangMigration.md
gogins commented 5 years ago

I am dropping LuaJIT from the Android app as there does not seem to be support for building LuaJIT with clang, which is now a requirement for the NDK.

gogins commented 5 years ago

TODOs now reduced to:

gogins commented 5 years ago

Run revised test list in the test-examples/README.md file. Done.

gogins commented 5 years ago

Inspect packages, remove any unwanted files, and upload.

gogins commented 5 years ago

Published the Csound for Android app. Published a new release of csound-aeolus. Published a new release of csound-extended-dev. Posted a new build of csound-vst to tumblr. Announced on tumblr, the electroacoustic composers group on Facebook, and to the csound, csound-dev, and cec-conference lists.