search

falkTX / Carla

Audio plugin host
https://kx.studio/carla
1.59k stars 145 forks source link

Please support FreeBSD #523

Closed yurivict closed 6 years ago

yurivict commented 6 years ago

Continuation from https://github.com/falkTX/Carla/issues/503

Here's what I think the easiest way to proceed: I will create the VirtualBox VM with FreeBSD and the suggested carla port that fails in build. You can just add this VM image into VirtualBox and take it from there.

Will this work for you?

falkTX commented 6 years ago

installing freebsd should not be that much of an issue, is it? I guess 64bit is the one to go for, right? do you do any kind of special setup for the compiler, or does clang-4.0 come as default?

yurivict commented 6 years ago

Yes, you can just install the amd64 FreeBSD from the ISO image, install all dependencies from packages (pkg install lv2 sratom ... etc), and build. GNU make is called gmake, and can be installed with pkg install gmake.

System compiler cc/c++ should suffice. clang-4.0 is the default.

The sample errors I am getting:

In file included from juce_core.cpp:146:
./network/juce_Socket.cpp:164:32: error: cannot initialize a parameter of type 'const struct sockaddr *' with an rvalue of type 'const juce::sockaddr *'
        return ::bind (handle, (const sockaddr*) &addr, sizeof (addr)) >= 0;
                               ^~~~~~~~~~~~~~~~~~~~~~~
In file included from juce_core.cpp:146:
./network/juce_Socket.cpp:223:83: error: cannot initialize a parameter of type 'struct sockaddr *' with an rvalue of type 'juce::sockaddr *'
                        bytesThisTime = ::recvfrom (handle, buffer, numToRead, 0, (sockaddr*) &client, &clientLen);
                                                                                  ^~~~~~~~~~~~~~~~~~~

Please let me know if you have any questions, I am happy to help.

falkTX commented 6 years ago

Something you can already do is check if dpf stuff builds there. See if you can build this project https://github.com/distrho/dpf-plugins

Carla assumes all OSes can build dpf based plugins. So it might be an issue during the build. Requires X11, everything else optional. Note: in the future external plugins won't be part of the default build, but I'm interested to know if those build there anyway.

falkTX commented 6 years ago

Oh for FreeBSD, what should be the support for Audio/MIDI? Carla has RtAudio included, which I guess builds there and can work using OSS. JACK is included on all platforms since carla provides custom headers + dlopen.

What about MIDI? We can use RtMidi, included in carla already. But are there any APIs to use on BSD systems?

(as you can probably tell, I'm really a noob when it comes to BSD stuff)

yurivict commented 6 years ago

Oh for FreeBSD, what should be the support for Audio/MIDI?

Jack should be the preferred audio interface. OSS is a basic sound interface on FreeBSD that always works. RtAudio might need to have ALSA disabled since ALSA isn't supported on FreeBSD.

What about MIDI? We can use RtMidi, included in carla already.

Jack supports MIDI. It can be further interpreted by fluidsynth (soft midi). OSS also supports MIDI, only USB soundcards support it, I believe.

yurivict commented 6 years ago

See if you can build this project https://github.com/distrho/dpf-plugins

It fails with missing mntent.h, which has different interface on FreeBSD. This is patchable. I will create the port for dpf-plugins, then Carla can just depend on the package.

falkTX commented 6 years ago

Where is mntent.h used in DPF code? Carla includes DPF source code, it's built together with its main code. Until the separation of external plugins is done, we need to be able to build DPF stuff (or we can just disable openGL, to disable DPF UI stuff)

yurivict commented 6 years ago

Where is mntent.h used in DPF code?

mntent.h is used in ./dpf/dgl/src/sofd/libsofd.c

Carla includes DPF source code, it's built together with its main code.

Bundling is bad because it requires such patching, can cause security problems, installation problems, etc.

falkTX commented 6 years ago

building it together is mostly for users who don't have any plugins installed. it only makes building harder if the included code doesn't support the target OS. nothing in DPF-Plugins uses sofd, so I can add a macro to optionally disable it during build.

Installation problems are exactly why the plugins are bundled together, it removes exactly that! I see no issues regarding security here. These plugins often don't read files and afaik never connect to online services. Worst they can do is crash when loaded, but that can happen if you load the plugins the normal way via shared libraries. Compared to loading random shared libs into the main process, built-in plugins are quite safe (we can actually check what the plugins are doing before building).

yurivict commented 6 years ago

Installation problems are exactly why the plugins are bundled together

Bundled parts tend to install into the same locations where the same packages are installed standalone, hence causing installation issues.

I see no issues regarding security here.

Maybe not here, but in general, when vulnerability is discovered in the bundled software it has to be patched in each bundled instance, many times, not just in one place.

In any case, please ask questions if you need help with FreeBSD.

falkTX commented 6 years ago

Juce has been removed, somewhat. I took pieces apart from an older juce version, to only have exactly what carla needs. This means there's no big juce libraries to be compiled, and removes a lot of platform independent code. There's no linux-specific code on this "fork" anymore, it only uses either win32 api or posix stuff (with minor exceptions for macOS, which doesn't have full posix support).

Please grab the latest code, in the no-juce branch, and see if you can get carla to build.

PS: I did try to install a freebsd minimal image, but then failed to get a desktop working. I'll install a full bsd + gui soon.

falkTX commented 6 years ago

no-juce branch is no longer needed, has been merged to master. I'm in the process of having the extra plugins being put into a separate repo.

If you pull the latest code and build using make EXTERNAL_PLUGINS=false you should be able to get a bit far in the build, perhaps even complete it. Please test.

falkTX commented 6 years ago

DPF build in Carla no longer tries to build sofd, as no plugins need it. This should make things easier to build on BSD, when including the internal plugins. @yurivict please give it try..

yurivict commented 6 years ago

Ok, thanks, I will try it this week.

yurivict commented 6 years ago

I am getting some compile errors and warnings with clang-4.0:

In file included from CarlaEngine.cpp:41:
../../modules/water/xml/XmlElement.h:647:36: error: no matching constructor for initialization of 'HeapBlock<water::XmlElement *>'
            HeapBlock<XmlElement*> elems ((size_t) num);
                                   ^      ~~~~~~~~~~~~
../../modules/water/xml/../memory/HeapBlock.h:81:7: note: candidate constructor (the implicit copy constructor) not viable: no known conversion from 'size_t' (aka 'unsigned long') to
      'const water::HeapBlock<water::XmlElement *, false>' for 1st argument
class HeapBlock
      ^
../../modules/water/xml/../memory/HeapBlock.h:103:5: note: candidate constructor not viable: no known conversion from 'size_t' (aka 'unsigned long') to
      'water::HeapBlock<water::XmlElement *, false>' for 1st argument
    HeapBlock (HeapBlock&& other) noexcept
    ^
../../modules/water/xml/../memory/HeapBlock.h:90:5: note: candidate constructor not viable: requires 0 arguments, but 1 was provided
    HeapBlock() noexcept  : data (nullptr)
    ^

CarlaPluginBridge.cpp:247:55: warning: format specifies type 'unsigned int' but the argument has type 'uintptr_t' (aka 'unsigned long') [-Wformat]
            std::snprintf(strBuf, STR_MAX, P_UINTPTR, options.frontendWinId);
                                                      ^~~~~~~~~~~~~~~~~~~~~
warning: unknown warning option '-Wno-misleading-indentation'; did you mean '-Wno-missing-declarations'? [-Wunknown-warning-option]

In file included from zynaddsubfx-ui.cpp:18:
../utils/CarlaPipeUtils.cpp:1066:17: warning: format specifies type 'unsigned int' but the argument has type 'std::size_t' (aka 'unsigned long') [-Wformat]
                size, ret, msg);
                ^~~~
../utils/CarlaPipeUtils.cpp:1066:23: warning: format specifies type 'int' but the argument has type 'ssize_t' (aka 'long') [-Wformat]
                size, ret, msg);
                      ^~~
falkTX commented 6 years ago

Thanks for testing. Added fixes for all of these now. Try again and let me know the results.

yurivict commented 6 years ago

Sorry, some more errors/warnings:

CarlaEngineRtAudio.cpp:404:48: error: member function 'getCurrentApi' not viable: 'this' argument has type 'const RtAudio', but function is not marked const
        return CarlaBackend::getRtAudioApiName(fAudio.getCurrentApi());
                                               ^~~~~~
/usr/local/include/rtaudio/RtAudio.h:829:32: note: 'getCurrentApi' declared here
inline RtAudio::Api RtAudio :: getCurrentApi( void ) { return rtapi_->getCurrentApi(); }
                               ^

RtAudio.cpp:7989:15: warning: address of 'info->doRealtime' will always evaluate to 'true' [-Wpointer-bool-conversion]
  if ( &info->doRealtime ) {
  ~~    ~~~~~~^~~~~~~~~~
ad_ffmpeg.c:223:13: warning: 'avcodec_decode_audio4' is deprecated [-Wdeprecated-declarations]
      ret = avcodec_decode_audio4(priv->codecContext, &avf, &got_frame, &priv->packet);
            ^
ad_ffmpeg.c:203:32: warning: 'av_free_packet' is deprecated [-Wdeprecated-declarations]
        if (priv->packet.data) av_free_packet(&priv->packet);
                               ^

You can also remove all register keywords across the project:

RtAudio.cpp:10053:3: warning: 'register' storage class specifier is deprecated and incompatible with C++1z [-Wdeprecated-register]
  register char *ptr;
  ^~~~~~~~~
falkTX commented 6 years ago

you're building against your local rtaudio? this won't work... (the carla provided version is patched, and will not give this error) which leads me to the question: why is /usr/local/include in the include paths? If you did not add anything manually, temporarily get rid of /usr/local/include/rtaudio (ie, rename) so we can continue with the build.

the ad_ffmpeg stuff can be ignored for now, just a small thing to support extra audio files. make HAVE_FFMPEG=false will build carla without it.

yurivict commented 6 years ago

you're building against your local rtaudio?

I didn't really make this choice. Can you ensure that the bundled version is used in the makefile?

temporarily get rid of /usr/local/include/rtaudio

Ok

the ad_ffmpeg stuff can be ignored for now

Ok. I am mostly concerned about errors.

falkTX commented 6 years ago

I was not even aware it would pick up the rtaudio from the system. a simple solution would be to rename the include, but anyway I'm mostly interested on getting this to work first.. are there any more errors?

yurivict commented 6 years ago

FreeBSD has no libdl, it's linux-only. For example, this

ifeq ($(UNIX),true)
LINK_FLAGS += -ldl
endif

is wrong. I am removing them because otherwise links fail.

It also prints some strange warnings like this: `zynaddsubfx/UI/VirKeyboard.fl:2: unknown version '1.0302'

yurivict commented 6 years ago

It builds through now. Breaks in install in this sed command: sed -i 's?X-PREFIX-X?/usr/local?' I can probably fix it.

sed -i 's?X-PREFIX-X?/usr/local?' \
    /usr/ports/audio/carla/work/stage/usr/local/bin/carla-single
sed: 1: "/usr/ports/audio/carla/ ...": extra characters at the end of p command

I have a lot of local patches. Besides making -ldl linux-only, CARLA_OS_FREEBSD should be added in source/includes/CarlaDefines.h, etc.

I can submit a pull request for this, if you want.

falkTX commented 6 years ago

interesting about libdl. Haiku does not have it either, though MacOS does.

Your install PREFIX seems wrong (?), are you mixing DESTDIR and PREFIX? PREFIX is usually just /usr or /usr/local, where DESTDIR is the place from which the package is built.

Is there a need for CARLA_OS_FREEBSD? Carla has no BSD specific code, it's just the fallback for generic POSIX systems.

You can attach a patch file directly, and I'll review the changes. I will likely not accept a direct PR for something I can't test myself.

yurivict commented 6 years ago

Your install PREFIX seems wrong (?), are you mixing DESTDIR and PREFIX?

Yes, this is how all ports do it: it is a concatenation of DESTDIR which is a stage directory, and PREFIX=/usr/local

Is there a need for CARLA_OS_FREEBSD?

There are several places, I will paste patches, you judge for yourself:

--- source/includes/CarlaDefines.h.orig 2017-11-18 21:07:07 UTC
+++ source/includes/CarlaDefines.h
@@ -46,6 +46,8 @@
 # define CARLA_OS_HAIKU
 #elif defined(__linux__) || defined(__linux)
 # define CARLA_OS_LINUX
+#elif defined(__FreeBSD__)
+# define CARLA_OS_FREEBSD
 #else
 # warning Unsupported platform!
 #endif
@@ -130,7 +132,7 @@
 #endif

 /* Define BINARY_NATIVE */
-#if defined(CARLA_OS_HAIKU) || defined(CARLA_OS_UNIX)
+#if defined(CARLA_OS_HAIKU) || defined(CARLA_OS_UNIX) || defined(CARLA_OS_FREEBSD)
 # ifdef CARLA_OS_64BIT
 #  define BINARY_NATIVE BINARY_POSIX64
 # else

--- source/modules/water/files/File.cpp.orig    2017-11-18 21:07:07 UTC
+++ source/modules/water/files/File.cpp
@@ -1196,7 +1196,7 @@ private:^M
 //=====================================================================================================================^M
 namespace^M
 {^M
-   #ifdef CARLA_OS_MAC^M
+   #if defined(CARLA_OS_MAC) ||  defined(CARLA_OS_FREEBSD)^M
     typedef struct stat   water_statStruct;^M
     #define WATER_STAT    stat^M
    #else^M

--- source/Makefile.mk.orig     2017-11-18 21:07:07 UTC
+++ source/Makefile.mk
@@ -26,10 +26,12 @@ CXX ?= g++
 ifneq ($(HAIKU),true)
 ifneq ($(MACOS),true)
 ifneq ($(WIN32),true)
+ifneq ($(FREEBSD),true)
 LINUX=true
 endif
 endif
 endif
+endif

 # --------------------------------------------------------------
 # Set MACOS_OR_WIN32
@@ -49,6 +51,10 @@ ifeq ($(LINUX),true)
 UNIX=true
 endif

+ifeq ($(FREEBSD),true)
+UNIX=true
+endif
+
 ifeq ($(MACOS),true)
 UNIX=true
 endif

@@ -153,6 +159,18 @@ else
 CXXFLAGS   += -isystem /usr/include/qt5
 endif
 endif
+ifeq ($(FREEBSD),true)
+BASE_FLAGS += -isystem $(PREFIX)/kxstudio/include
+CXXFLAGS   += -isystem $(PREFIX)/glib-2.0
+CXXFLAGS   += -isystem $(PREFIX)/glib-2.0/glib
+CXXFLAGS   += -isystem $(PREFIX)/gtk-2.0
+CXXFLAGS   += -isystem $(PREFIX)/gtk-2.0/gio
+ifeq ($(DEFAULT_QT),4)
+CXXFLAGS   += -isystem $(PREFIX)/include/qt4
+else
+CXXFLAGS   += -isystem $(PREFIX)/include/qt5
+endif
+endif
 ifeq ($(MACOS),true)
 BASE_FLAGS += -isystem /opt/kxstudio/include
 CXXFLAGS   += -isystem /System/Library/Frameworks

--- source/utils/CarlaSemUtils.hpp.orig 2017-11-18 21:07:07 UTC
+++ source/utils/CarlaSemUtils.hpp
@@ -22,7 +22,7 @@

 #include <ctime>

-#ifndef CARLA_OS_HAIKU
+#if !defined(CARLA_OS_HAIKU) && !defined(CARLA_OS_FREEBSD)
 # define CARLA_USE_FUTEXES
 #endif

--- source/utils/CarlaShmUtils.hpp.orig 2017-11-18 21:07:07 UTC
+++ source/utils/CarlaShmUtils.hpp
@@ -27,7 +27,7 @@ struct carla_shm_t { HANDLE map; bool is
 # ifndef __WINE__
 #  include <cerrno>
 # endif
-# ifdef CARLA_OS_HAIKU
+# if !defined(CARLA_OS_HAIKU)
 #  define MAP_LOCKED 0x0
 # endif
 # include <fcntl.h>
@@ -36,6 +36,10 @@ struct carla_shm_t { int fd; const char*
 # define carla_shm_t_INIT { -1, nullptr, 0 }
 #endif

+# if defined(CARLA_OS_FREEBSD)
+#  define MAP_LOCKED 0x0
+# endif
+
yurivict commented 6 years ago

For -ldl, IMO, you should just put it in one place, like add it in the beginning of the top Makefile, so that it will be easy to patch. Right now there are too many patches for -ldl.

falkTX commented 6 years ago

The patch seems quite okay, though I'd use BSD instead of FREEBSD, we there's other variants of BSD out there. Some of the code about !haiku && !bsd can change to if-linux I guess. And correct about libdl, I should define it in a single place.

I'll commit something soon and leave a comment when done.

yurivict commented 6 years ago

I'll commit something soon and leave a comment when done.

Thanks!

yurivict commented 6 years ago

FYI: To catch all BSDs, you need to have

if defined(__FreeBSD__) || defined(__NetBSD__) || defined(__DragonFly__) || defined(__OpenBSD__)
falkTX commented 6 years ago

I pushed a few commits now, please use make BSD=true and let me know how it goes. I'm curious to know how carla actually runs...

yurivict commented 6 years ago

It builds fine now. The correct sed syntax is sed -i '' -e {regexp} , then it installs too.

Running carla-single produces the usage message. How to see the GUI screen?

For some reason, DEFAULT_QT=5 doesn't make anything depend on Qt5. Qt4 dependencies exist (from lib/carla/carla-bridge-lv2-qt4). If Qt4/Qt5 is for plugin UI support, why have DEFAULT_QT?

falkTX commented 6 years ago

DEFAULT_QT is to choose what the frontend will use. It can be either Qt4 or Qt5. (frontend being PyQt4 or PyQt5)

The extra toolkits are there for LV2 UI support. Since, for example, guitarix and invada plugins are gtk2 based, Carla needs some gtk2 code in order to show their UIs. Same applies for gtk3, Qt4 and Qt5.

If you only have carla-single and not the main 'carla' script available, then the frontend was not built. Run make features to see what else might be missing. (After installing a new optional dependency you need to rebuild carla)

My guess is that you're missing pyqt now. (needs python3 version) Or you have to define the path to pyuic4/5, like: make BSD=true DEFAULT_QT=5 PYUIC5=/usr/local/bin/pyuic5

By default it detects if /usr/bin/pyuic5 exists. Maybe for BSD the path is different...?

yurivict commented 6 years ago

Run make features to see what else might be missing.

While I am looking why some features aren't detected, please apply this patch to Makefile:

-USE_COLORS=true
-
-ifeq ($(HAIKU),true)
-USE_COLORS=false
-endif
+USE_COLORS=$(shell [ -n "$(MAKE_TERMOUT)" ] && echo true)

 ifeq ($(USE_COLORS),true)
 ANS_NO=\033[31m NO \033[0m
@@ -739,179 +736,179 @@ FEV="Qt5"
 endif

 features_print_main:
-       @echo "$(tS)---> Main features $(tE)"
+       @echo -e "$(tS)---> Main features $(tE)"

Please replace all @echo commands with @echo -e in Makefile and source/native-plugins/external/Makefile.mk



-----
It always prints terminal sequences, which is very ugly.
yurivict commented 6 years ago

In source/Makefile.mk, you need to replace $(wildcard $(PYUIC5)) with just $(PYUIC5). wildcard doesn't work for some reason, and is also not needed there. Other similar places need such replacements too: PYUIC4, MOC_QT4, etc.

source/Makefile.mk also has wrong lines:

PYUIC4 ?= /usr/bin/pyuic4
PYUIC5 ?= /usr/bin/pyuic5

they make these values always set and default to Qt4.

LINUX=true on BSD by mistake. This allows ALSA detection, which generally doesn't exist on BSD.

falkTX commented 6 years ago

the echo stuff is something I still have to do. people usually just tell how to do it, but not the actual reason why... when I change it, I need to understand why.

regarding makefile wildcards, they're are on purpose and won't be removed. they are not meant to check if a variable is defined, but they actually test if the filename exists on the system. that's how /usr/bin/pyuic5 is detected if it exists or not via makefile.

yurivict commented 6 years ago

people usually just tell how to do it, but not the actual reason why... when I change it, I need to understand why.

Because currently it prints terminal sequences in a mangled way, not as colors, and regardless of whether you are in the terminal or not.

The patch above fixes both problems:

regarding makefile wildcards, they're are on purpose and won't be removed.

wildcards resolve to empty value for some reason, while the file exists.

falkTX commented 6 years ago

correct about wildcards, that's why there's a negative check when we want to verify if something exists. if wildcard returns negative/empty, the file exists.

yurivict commented 6 years ago

I am talking about this place:

ifneq (,$(wildcard $(PYUIC5)))
HAVE_PYQT=true
HAVE_PYQT5=true
else
HAVE_PYQT5=false
endif

PYUIC5 is set to /usr/local/bin/pyuic5, but $(wildcard $(PYUIC5)) evaluates to an empty value, and HAVE_PYQT5 becomes false.

falkTX commented 6 years ago

hmm you're reading it wrong. there's a very clear ifneq in there. if the wildcard returns empty, HAVE_PYQT5 will be true.

yurivict commented 6 years ago

wildcard is empty, the compared value is empty, it should be ifeq for HAVE_PYQT5=true.

falkTX commented 6 years ago

if wildcard is empty and you do ifeq on it, it will return false. I want to have the most relevant result first, so I invert the condition. not sure why this is hard to understand...

yurivict commented 6 years ago

I can't understand why is wildcard empty when the file exists. Wildcard is supposed to expand the pattern, for the existing file it should just return this file, not an empty value.

In any case, carla fails to find PyQt5 even though I pass a valid /usr/local/bin/pyuic5.

yurivict commented 6 years ago

Maybe I don't get the logic, but PyQt5 should be found when a valid PYUIC5 is passed, and it isn't found. So the net result is wrong.

falkTX commented 6 years ago

that's just how wildcards work in makefiles... the makefiles are been setup and working on linux, macOS and haiku. maybe bsd uses a different program to process makefiles? (no idea really)

for now I guess you can force more makefile variables, as we first need to see how carla even runs there... something like: make BSD=true DEFAULT_QT=5 HAVE_PYQT=true HAVE_PYQT5=true

yurivict commented 6 years ago

No, the same GNU make is used.

yurivict commented 6 years ago

Here is how make features output looks like without the patch:

carla-terminal-sequences

falkTX commented 6 years ago

We can only the terminal print stuff later on. (Haiku has the same issue, so I can test this myself) Let's focus on getting carla working on BSD first.

btw, how come you have ALSA and PulseAudio? weren't you using freebsd for building carla...

yurivict commented 6 years ago

btw, how come you have ALSA and PulseAudio?

This was caused by mistaken arguments.

It builds, but there is an assortment of small problems.

  1. It should include BSD.
  2. Why is it disabled when HAVE_QT5=true?

Here https://github.com/falkTX/Carla/blob/master/source/Makefile.mk#L193 you probably want to have ?= in order to respect the outside settings.

falkTX commented 6 years ago

Why can't a port have both qt4 and qt5? They're never used in the same address space, so that part is not an issue. Losing one of them means we drop support for a specific lv2 ui type...

The Qt4 and Qt5 detection is made by pkg-config, I see no issues with that. If something is possible to build, we build it..

There are some things I assumed were Linux only, now corrected.

yurivict commented 6 years ago

Why can't a port have both qt4 and qt5?

This is a limitation of the framework: it sets options, paths, etc only for one Qt version. Also, Qt4 is simply going away, replaced with Qt5.

Losing one of them means we drop support for a specific lv2 ui type...

I know, I just want to make the port work for now. Most LV2 UIs I saw are Gtk for some reason. If I ask for dual Qt4/Qt5 support, people will likely ask: Why isn't Qt4 code ported to Qt5?

Ok, thanks, I will update the revision and see what else is missing.

falkTX commented 6 years ago

Understandable about Qt5. I guess if rncbc synths and qmidiarp get ported to freebsd, they'll use the qt5 version. So having only Qt5 should be ok.

On local builds you might get Qt4 built along as well, but this won't influence things too much. (Unless the default qt goes to qt4? I should change that by now..)