falkTX / Carla

Audio plugin host
https://kx.studio/carla
1.59k stars 145 forks source link

Please support FreeBSD #523

Closed yurivict closed 6 years ago

yurivict commented 6 years ago

Continuation from https://github.com/falkTX/Carla/issues/503

Here's what I think the easiest way to proceed: I will create the VirtualBox VM with FreeBSD and the suggested carla port that fails in build. You can just add this VM image into VirtualBox and take it from there.

Will this work for you?

falkTX commented 6 years ago

ok, qt4 was being used by default if both pyqt4 and pyqt5 were available. now changed to qt5 as default.

yurivict commented 6 years ago

It generally works on FreeBSD.

The problem I noticed is that when some LV2 plugin is instantiated, the "New" menu command or "Remove All" button hang the app.

Another problem is that the patchbay leaves drawing artifacts when you move the nodes, see the screenshot:

carla-drawing-artifacts

Otherwise, I can create a port for Carla.

falkTX commented 6 years ago

Locking the app can happen if the audio stream stops running. It's something I have to fix, and finally I am able to reproduce it easily with the Dummy RtAudio option.

Artifacts on canvas come from Qt, not much I can do except trying to mess around with optimization and cache flags for Qt GraphicsScene/View stuff. Though it often happens if you move 2 boxes at once.

Good to know that Carla is running :) I was finally able to get a freebsd VM running, though without X11 for now, just seeing if everything builds

yurivict commented 6 years ago

You should also replace #!/bin/bash with #!/bin/sh in all scripts under bin/. Using the Bourne shell eliminates the bash dependency on BSD.

falkTX commented 6 years ago

I can convert them sure, but need to make sure the actual syntax used on those files is compatible with the simple /bin/sh shell.

yurivict commented 6 years ago

need to make sure the actual syntax used on those files is compatible with the simple /bin/sh shell.

It is compatible.

yurivict commented 6 years ago

You probably don't need to install the C++ file: include/carla/utils/CarlaPipeUtils.cpp Also not sure if headers need to be installed.

yurivict commented 6 years ago

The FreeBSD port works. At this point, I am trying to simplify large patches to eliminate questions about them when I submit the port:

yurivict commented 6 years ago

Additionally, please consider changing absolute symbolic links to relative links. Currently, I get warnings:

Warning: Bad symlink '/usr/local/lib/lv2/carla.lv2/resources' pointing to an absolute pathname '/usr/local/share/carla/resources'

While the warning itself is questionable, it is probably better to have links pointing to something like ../../../share/carla/resources.

FYI: The current port version: https://reviews.freebsd.org/D13172 - all patches are there.

falkTX commented 6 years ago

Ok some notes:

  1. @echo -e does not work on linux, I get this output in make features:
    -e ---> Main features 
    -e Front-End:    YES (Using Qt5)
    -e LV2 plugin:   YES 
    -e VST plugin:   YES 
    -e Link support: YES 
    -e OSC support:  YES

    I guess this is because the echo used in makefiles is a special command, not the actual system echo from the system. On BSD the echo -e works as-is... I'll have to investigate.

In any case, I added the check for MAKE_TERMOUT now. Also startup scripts use /bin/sh instead of /bin/bash.

Regarding the bsd port files, some notes:

  1. Is there a way to specify optional dependencies? Some like ffmpeg should really be optional, default being off
  2. Please remove fftw3, mxml, libprojectm and ntk. They are used by internal plugins, which will not be part of the default build soon. You can get LV2 and VST versions of all these plugins separately, which is preferred for source-based builds.
  3. DEFAULT_QT=5 can probably be removed now
  4. EXPERIMENTAL_PLUGINS=false is not needed, it's always false by default and I will likely remove this option soon
  5. BUILD_DEPENDS on lv2 is wrong, Carla includes all LV2 headers that it needs
  6. RUN_DEPENDS on jack is wrong. Carla loads libjack dynamically, it won't fail if libjack is not installed.

I don't like the way you're patching source/Makefile.mk file, but I'll have to find a way to make the checks work for BSD first. So for now it's fine I guess. Why is pulseaudio check forced to true? There's a pkg-config check for this, which is not linux specific.

yurivict commented 6 years ago

Then I don't know a better solution than patching for @echo -e.

  1. Is there a way to specify optional dependencies? Some like ffmpeg should really be optional, default being off

Yes, there are port options. I added the port option.

  1. Please remove fftw3, mxml, libprojectm and ntk. Some like ffmpeg should really be optional, default being off

It goes by what executables actually depend on. Once executables will lose these dependencies I will drop them.

  1. RUN_DEPENDS on jack is wrong.

I made it optional, and updated the rest.

falkTX commented 6 years ago

Then I ask you to add EXTERNAL_PLUGINS=false during make, this will skip the internal plugins that I will make optional later on. (it's a work in progress, need to setup a new git repo as submodule etc etc) So with this Carla lose the dependency on ntk, fftw3 and a bunch of other stuff. (OpenGL headers will not be needed either)

yurivict commented 6 years ago

With EXTERNAL_PLUGINS=false it still links with ntk, projectM, mxml, fftw3.

falkTX commented 6 years ago

...did you do a clean build?

yurivict commented 6 years ago

...did you do a clean build?

Always.

falkTX commented 6 years ago

Oh I think I know what's going on! The way you're running make that you need the "?=" means you cannot override makefile variables. Instead of using something like: BSD=true HAVE_QT4=false make please use this instead: make BSD=true HAVE_QT4=false

This way you don't need to patch the makefile anymore, at least regarding "?=" usage.

I don't like to read env vars on the makefile, as users might have them defined to random things which breaks the build in random ways. Forcing to use variables after the make command solves this.

yurivict commented 6 years ago

Oh I think I know what's going on!

That was it! I updated the port.

falkTX commented 6 years ago

Looking better now.

Some extra things:

  1. Please remove FFMPEG_MAKE_ARGS= HAVE_FFMPEG=true, the pkg-config check should be enough. We need pkg-config in order to get the build and link flags from ffmpeg anyway, so forcing this is not a good idea.
  2. Similar to # 1 please remove HAVE_PULSEAUDIO=true
  3. Remove the "?=" patching on the makefile, not needed anymore
  4. Make PYUIC5 and PYRCC5 env vars, like we had before. This is one of the few variables where I think env var is a good idea, since the path to these tools is not the same across all systems.
  5. See if you can remove HAVE_PYQT=true and HAVE_PYQT5=true. The reason is that PYUIC5 will evalute as non-empty, so these 2 should be getting set.

oh, and can we have linuxsampler as dependency? it would provide gig and sfz file support.

yurivict commented 6 years ago

Please remove FFMPEG_MAKE_ARGS= HAVE_FFMPEG=true, the pkg-config check should be enough

No, you are wrong. Port option should drive the choice, not pkg-config. Package might be installed, and found by pkg-config, but if port option is "off" it should be ignored.


After some of the modifications, now these files aren't built and installed:

Error: Missing: share/carla/resources/zynaddsubfx-ui
Error: Missing: share/carla/resources/zynaddsubfx/black_key.png
Error: Missing: share/carla/resources/zynaddsubfx/black_key_pressed.png
Error: Missing: share/carla/resources/zynaddsubfx/knob.png
Error: Missing: share/carla/resources/zynaddsubfx/module_backdrop.png
Error: Missing: share/carla/resources/zynaddsubfx/white_key.png
Error: Missing: share/carla/resources/zynaddsubfx/white_key_pressed.png
Error: Missing: share/carla/resources/zynaddsubfx/window_backdrop.png

I am not sure what change exactly causes this.

falkTX commented 6 years ago

You misunderstand me. When ffmpeg option is off, then you keep the HAVE_FFMPEG=false, to forcedly disable ffmpeg. When ffmpeg option is on, then it will require ffmpeg to be installed, which then it will be found by pkg-config and the makefile. So you only remove the HAVE_FFMPEG=true, which works out fine afaik.

Because ffmpeg keeps updating and changing, I will need to restrict the version supported. Someone with a newer ffmpeg might cause the build to break. If you force a package test to be true, then you also need to ensure the code always builds. Are you going to keep checking if carla builds for each ffmpeg update? Please remove the forced HAVE_FFMPEG=true, and let pkg-config handle it.

The extra files came from the extra internal plugins (which now are disabled), so it's normal they're not available anymore.

yurivict commented 6 years ago

You misunderstand me.

Got it now! I updated the port.

falkTX commented 6 years ago

Thanks.

Small typo there, env var for PYUIC should be PYUIC5, same for PYRCC5

Also I suggest not patching the makefile wildcard stuff for qt4, since it's never used anyway. This will make the patch contents smaller and easier to maintain.

yurivict commented 6 years ago

Also, please remove all *.orig files. They conflict with the patching process. -)

I fixed the typo.

Also I suggest not patching the makefile wildcard stuff for qt4

But they are evaluated the wrong way. What if they wrongly enable Qt4?

falkTX commented 6 years ago

They can never enable qt4 since you force it to off now, using the make HAVE_QT4=false, it will override the checks there

Also I see @dir lib/carla/jack so you can probably use the same for share/carla, include/carla and lib/carla and lib/lv2/carla.lv2.

yurivict commented 6 years ago

Ok, updated the patch.

@dir instructions are auto-generated. They are usually ok as they are generated. -)

yurivict commented 6 years ago

I think the port is in a much better shape now.

You need to let build accept the PYTHON variable, and write it into all bin/carla* scripts: PYTHON="/path/to/python". This will further simplify the port. Default it to $(which python) IMO.

I am off to bed for today.

Thank you for your help with the port!

falkTX commented 6 years ago

which python does not work because some distros use python2 by default. best I can do is run which python3 and see if that exists. then fallback to just python.

There's still a few things:

  1. PulseAudio needs to be an optional built-time dependency, not runtime. Unlike JACK, the PulseAudio stuff needs to be there during build in order to be enabled
  2. USE_GL can be removed
  3. I suggest splitting USE_GNOME, and rename it to USE_LV2_GTK2 and USE_LV2_GTK3

The rest I guess I have to do myself, for me I guess it is:

  1. figure out better way to find and use python3
  2. fix echo -e somehow
  3. fix install sed commands
falkTX commented 6 years ago

Colored echo is fixed now.

falkTX commented 6 years ago

btw, somehow on my VM Qt5 shows as disabled even though it's there... pkg-config fails to find it for some reason, that's why the check for MOC_QT5 needs to be forced. I think we need HAVE_QT5=true after the makefile for now.

Also my system is missing pyrcc5, not sure what's going on.

falkTX commented 6 years ago

pyuic5 and pyrcc5 is now automatically detected, no need to specify via env vars.

falkTX commented 6 years ago

and the make install should work now. only thing left is the startup scripts to use python3 in a better way. (sorry for all the messages, but I have been doing the things slowly through the day as I had time)

yurivict commented 6 years ago

No problem about many messages!

Also my system is missing pyrcc5, not sure what's going on.

You need to place the port under /usr/ports (it will be /usr/ports/audio/carla), run make, and it will rebuild all dependencies.

Colored echo is fixed now.

Thanks!


I updated the port.


I recommend to create one shell executable for all of data/carla-*, for example in share/carla/cmd.sh, and in data/carla-* call it like this:

#!/bin/sh

%%PREFIX%%/share/carla/cmd.sh jack-multi "$@"

Then python selection will be only in this one file.

falkTX commented 6 years ago

You forgot the part that make-install is fixed now.

Also a few things more:

  1. ffmpeg uses LIB_DEPENDS while pulseaudio uses BUILD_DEPENDS, why are they different?
  2. USE_PYQT has xml tag, but carla never uses QtXml stuff
  3. Deleting "*.orig" files no longer needed, I removed those from the repo
  4. fluidsynth can also be made an optional dependency
  5. again, can linuxsampler be added as optional dependency? (or not? then why?)
  6. does bsd-ports provide python3-liblo? if so, we should add it as depends, so that carla-control also works
yurivict commented 6 years ago

I updated for all 6 points. Yes, python3-liblo is supported now (with my patch https://reviews.freebsd.org/D13189).


Now there are 5 problems:

  1. Library linkage problems:
    • Nothing links to libpulse-simple.so when PULSEAUDIO=on, even though make features says that it is "on"
    • linuxsampler isn't found. I have linuxsampler-2.0.0, pkg-config --libs --atleast-version=1.0.0.svn41 linuxsampler doesn't find it for some reason.
  2. Python executable selection should be moved into one file (see my suggestion about cmd.sh)
  3. Warnings about absolute symlinks need to be fixed (make symlinks relative):

    Warning: Bad symlink '/usr/local/share/carla/resources/ui_carla_plugin_default.py' pointing to an absolute pathname '/usr/local/share/carla/ui_carla_plugin_default.py'

  4. -i argument of sed expects the string parameter. Therefore, the patch is still there. -i tells it to do the in-place substitution. The parameter is the extension to rename the original into. In Linux, obviously, there's no parameter.
  5. Build on i386 system fails:
    In file included from JackBridge2.cpp:22:
    ../utils/CarlaSemUtils.hpp:224:24: error: non-constant-expression cannot be narrowed from type 'uint' (aka 'unsigned int') to 'time_t' (aka 'int') in initializer list [-Wc++11-narrowing]
    timespec delta = { secs, nsecs };
                       ^~~~
    ../utils/CarlaSemUtils.hpp:224:24: note: override this message by inserting an explicit cast
    timespec delta = { secs, nsecs };
                       ^~~~
                       static_cast<time_t>( )
    ../utils/CarlaSemUtils.hpp:224:30: error: non-constant-expression cannot be narrowed from type 'uint' (aka 'unsigned int') to 'long' in initializer list [-Wc++11-narrowing]
    timespec delta = { secs, nsecs };
                             ^~~~~
    ../utils/CarlaSemUtils.hpp:224:30: note: override this message by inserting an explicit cast
    timespec delta = { secs, nsecs };
                             ^~~~~
                             static_cast<long>( )
    2 errors generated.
falkTX commented 6 years ago

Ok let's see... 1a. pulseaudio handled now. though rtaudio calls it linux-pulseaudio, so not sure if it's going to work as-is or not. we just try it 1b. it might be because linuxsampler doesn't come from the official repos, but ports. see where it places the *.pc files. my guess is that we need to define the PKG_CONFIG_PATH env var for pkg-config to find it

2 and 3 I will handle soon. I need to make it in a way that doesn't break existing setups, so I want to be careful.

  1. I am a little confused about the "-i" parameter. can we use sed -i -e <regexp> <file> with bsd sed? I remember on MacOS I never needed to supply a parameter for the inline stuff, it should be optional. So I believe sed -i -e ... should work. Please confirm.

  2. fixed now, I believe.

yurivict commented 6 years ago

we need to define the PKG_CONFIG_PATH

On FreeBSD all .pc files are in one directory. It does find it when --atleast-version=1.0.0.svn41 is removed.

So I believe sed -i -e ... should work. Please confirm.

Yes, this works.

  1. fixed now, I believe-

confirmed

falkTX commented 6 years ago

Alright, so probably the linuxsampler provided by bsd ports is still version 1.0.0? If not, then something is weird with the version check. Please test if using --atleast-version=2.0.0 works.

The sed make install stuff should work now. Let me know if building with pulseaudio works or not.

yurivict commented 6 years ago

linuxsampler is recognized now, not sure what has changed. I reinstalled it.

yurivict commented 6 years ago

@falkTX Sorry, sed -i -e also doesn't work, it treats -e as suffix, so -i really requires that parameter. Removing -i -e altogether works though. Does this work on Linux?

This solves the problem:

ifeq ($(BSD),true)
SED_ARGS=-i '' -e
else
SED_ARGS=-i -e
endif

// ...
sed $(SED_ARGS) 's?X-PREFIX-X?$(PREFIX)?' \
// ...
falkTX commented 6 years ago

What about this: sed -e <regexp> -i -- <file>

Strange that -i needs an argument, MacOS uses the same sed I believe and works fine without it.

yurivict commented 6 years ago

sed -e -i --

This leaves the extra file abc--. Use variable like in the previous message. I verified, it works in make.


MacOS was originally forked from FreeBSD, but it has been heavily modified over the years. I don't use MacOS, so can't tell more.

falkTX commented 6 years ago

Ok, I added that just now. I also fixed the Qt5 detection. The wildcard thing was working as intended, it was the paths that were not being setup correctly. So please remove HAVE_QT5=true, MOC_QT5=, RCC_QT5= and the makefile patch from the port, it should work as-is right now.

yurivict commented 6 years ago

Updated, thanks.

falkTX commented 6 years ago

The 2 final things to do are complete now. make-install now uses relative paths for the symlinks and startup scripts were tweaked to find python3 in a better way. Let me know if anything else is needed.

I removed the installation of some files, which are not needed anymore.

On the port file, please remove 'xml_build' from the pyqt flags. No python XML stuff is ever used.

yurivict commented 6 years ago

Thanks, I updated the port. It looks good now.

xml_build is needed because pyrcc5 lives there.

falkTX commented 6 years ago

Alright, we have a working port then :) Small final adjustment, I removed the $libdir/python3/dist-packages folder for non linux systems

As a final request, can you make a sorta-fancy/working carla screenshot on freebsd? Having this working is news worthy!^-^)/

yurivict commented 6 years ago

Thank you very much for your help with the port!

As a final request, can you make a sorta-fancy/working carla screenshot on freebsd? Having this working is news worthy!^-^)/

Sure, I will attach the screenshot tomorrow.

falkTX commented 6 years ago

Thanks for your help too. I am closing the issue now.

yurivict commented 6 years ago

Carla on FreeBSD screenshot: carla-on-freebsd

yurivict commented 6 years ago

@falkTX The Carla port is now committed into the repository. If you need to add the installation instructions to the HOWTO, on FreeBSD it is pkg install Carla.