Screen Capture failed to initialize

[nzoschke]

On 0.7.1 on OS X, screen capture always goes loading... then warning popup with Failed to initialize.

Full logs:

0050  Device Screen capture selected its optimal config: video/x-raw BGRA 3024x1964@30.0fps
0051  Stream 1232612263997249 Opened 'avfvideosrc capture-screen=true ! video/x-raw,format=BGRA,framerate=30/1,width=3024,height=1964 ! videoconvert ! video/x-raw,format=RGB ! queue max-size-buffers=3 ! videoconvert ! appsink name=sink' (3024 x 1964)
0052   Warning - Stream 1232612263997249 Failed to initialize.

I'm hoping to capture the screen from another program, projectM visualizer to layer in as an audio reactive element.

[nzoschke]

Very impressive project BTW!

[brunoherbelin]

Thanks!

This is very probably the same problem than in #58: The shared lib used for access to screen capture in OSX is a gstreamer plugin using AV Foundation (https://developer.apple.com/av-foundation/) ; it might require specific support for M1/M2 new architectures.

You are right to try to compile vimix for M1 specifically; let's see how this goes in #58 !

[nzoschke]

I did have luck building on M1 on https://github.com/brunoherbelin/vimix/issues/58! H

However then I get a new error:

Add Source -> Connected -> Select Stream

0001  Refusing stream requests to vimix@studio.attlocal.net.0. No streaming ongoing.
0002  Metronome Ableton Link Enabled
0003  Metronome Ableton Link start & stop sync Enabled
0004  No GPU decoding plugin found.
0005  OSC: Loaded 1 translation.
0006  OSC: Listening to UDP messages sent to 192.168.1.161:7000
0007  Font size 20
0008  New session ready.
0009   Session '' loaded with no source.
0010   Warning - DeviceSource Could not construct test pipeline avfvideosrc capture-screen=true name=devsrc ! fakesink name=sink:
no element "avfvideosrc"

[nzoschke]

Google sounds like more build or gstreamer quirks...

• https://github.com/Homebrew/homebrew-core/issues/109283
• https://gstreamer.freedesktop.org/documentation/applemedia/avfvideosrc.html?gi-language=c
• https://stackoverflow.com/questions/64073844/no-element-avfvideosrc-in-gstreamer
• https://stackoverflow.com/questions/66296668/gstreamer-macos-big-sur-camera-input

[nzoschke]

I'm trying to install gstreamer via https://gstreamer.freedesktop.org/documentation/installing/on-mac-osx.html?gi-language=c#

But not yet getting it wired up...

diff --git a/CMakeLists.txt b/CMakeLists.txt
index f200d85..ae6f16c 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -59,7 +59,7 @@ if(UNIX)
         # the RPATH to be used when installing
         set(CMAKE_SKIP_RPATH TRUE)
         set(OpenGL_DIR /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/)
-        set(CMAKE_OSX_ARCHITECTURES "x86_64")
+        set(CMAKE_OSX_ARCHITECTURES "arm64")
         set(CMAKE_OSX_DEPLOYMENT_TARGET "10.14")

         # CPACK
@@ -69,7 +69,8 @@ if(UNIX)
         set(APPLE_CODESIGN_ENTITLEMENTS "${CMAKE_CURRENT_SOURCE_DIR}/osx/entitlements.plist")

         # find icu4c in OSX (pretty well hidden...)
-        set(ENV{PKG_CONFIG_PATH} "$ENV{PKG_CONFIG_PATH}:/usr/local/opt/icu4c/lib/pkgconfig")
+        set(ENV{PKG_CONFIG_PATH} "$ENV{PKG_CONFIG_PATH}:/opt/homebrew/opt/icu4c/lib/pkgconfig:/Library/Frameworks/GStreamer.framework/Libraries/pkgconfig/")
+        set(ENV{PATH} "$ENV{PATH}:/Library/Frameworks/GStreamer.framework/Commands/")

     else()
         add_definitions(-DLINUX)

% cmake -DCMAKE_BUILD_TYPE=Release ../vimix           
-- Compiling vimix version 0.7.2
-- Could NOT find GSTREAMER_BASE_LIBRARY
-- Found GSTREAMER_BASE_INCLUDE_DIR: /Library/Frameworks/GStreamer.framework/Headers
-- Could NOT find GStreamer (missing: GSTREAMER_LIBRARY GSTREAMER_BASE_LIBRARY) (Required is at least version "1.0.0")
-- Could NOT find GSTREAMER_APP_LIBRARY
-- Found GSTREAMER_APP_INCLUDE_DIR: /Library/Frameworks/GStreamer.framework/Headers
-- Could NOT find GSTREAMER_AUDIO_LIBRARY
-- Found GSTREAMER_AUDIO_INCLUDE_DIR: /Library/Frameworks/GStreamer.framework/Headers
-- Could NOT find GSTREAMER_VIDEO_LIBRARY
-- Found GSTREAMER_VIDEO_INCLUDE_DIR: /Library/Frameworks/GStreamer.framework/Headers
-- Could NOT find GSTREAMER_PBUTILS_LIBRARY
-- Found GSTREAMER_PBUTILS_INCLUDE_DIR: /Library/Frameworks/GStreamer.framework/Headers
-- Could NOT find GSTREAMER_GL_LIBRARY
-- Found GSTREAMER_GL_INCLUDE_DIR: /Library/Frameworks/GStreamer.framework/Headers
-- Could NOT find GStreamerPluginsBase (missing: GSTREAMER_APP_LIBRARY GSTREAMER_AUDIO_LIBRARY GSTREAMER_VIDEO_LIBRARY GSTREAMER_PBUTILS_LIBRARY GSTREAMER_GL_LIBRARY) (Required is at least version "1.0.0")

[nzoschke]

Maybe I need to wire this through

https://gstreamer.freedesktop.org/documentation/installing/on-mac-osx.html?gi-language=c#manual-compilation-with-pkgconfig

[nzoschke]

Hm I went back to homebrew gstreamer since things are building there. I also see it working on its own with

brew install gstreamer gst-plugins-base gst-plugins-good gst-plugins-bad gst-plugins-ugly gst-libav gst-rtsp-server gst-editing-services gst-devtools
gst-launch-1.0 videotestsrc ! autovideosink

But still vimix has the warning:

0011   Warning - Stream 1313492662361541 Could not create stream: no element "videotestsrc".

[nzoschke]

This looks suspicious

 % ack gst
CMakeCache.txt
273:GSTREAMER_APP_LIBRARY:FILEPATH=/opt/homebrew/lib/libgstapp-1.0.dylib
279:GSTREAMER_AUDIO_LIBRARY:FILEPATH=/opt/homebrew/lib/libgstaudio-1.0.dylib
285:GSTREAMER_BASE_LIBRARY:FILEPATH=/opt/homebrew/lib/libgstbase-1.0.dylib
291:GSTREAMER_GL_LIBRARY:FILEPATH=/opt/homebrew/lib/libgstgl-1.0.dylib
297:GSTREAMER_LIBRARY:FILEPATH=/opt/homebrew/lib/libgstreamer-1.0.dylib
303:GSTREAMER_PBUTILS_LIBRARY:FILEPATH=/opt/homebrew/lib/libgstpbutils-1.0.dylib
305://The path to the gstreamer plugins installation directory
306:GSTREAMER_PLUGIN_DIR:PATH=/non_existent_on_purpose/please_use_framework_pkg-config/or_pass_--define-prefix_to_your_pkg-config/lib/gstreamer-1.0

[nzoschke]

I'm thinking gst-plugin-scanner /non_existent_on_purpose means it's building but cant scan for plugins, hence the runtime error. I tried more PKG_CONFIG_PATH hacking for homebrew but no luck yet!

% git diff
diff --git a/CMakeLists.txt b/CMakeLists.txt
index f200d85..77658b8 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -59,7 +59,7 @@ if(UNIX)
         # the RPATH to be used when installing
         set(CMAKE_SKIP_RPATH TRUE)
         set(OpenGL_DIR /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/)
-        set(CMAKE_OSX_ARCHITECTURES "x86_64")
+        set(CMAKE_OSX_ARCHITECTURES "arm64")
         set(CMAKE_OSX_DEPLOYMENT_TARGET "10.14")

         # CPACK
@@ -69,8 +69,7 @@ if(UNIX)
         set(APPLE_CODESIGN_ENTITLEMENTS "${CMAKE_CURRENT_SOURCE_DIR}/osx/entitlements.plist")

         # find icu4c in OSX (pretty well hidden...)
-        set(ENV{PKG_CONFIG_PATH} "$ENV{PKG_CONFIG_PATH}:/usr/local/opt/icu4c/lib/pkgconfig")
-
+        set(ENV{PKG_CONFIG_PATH} "$ENV{PKG_CONFIG_PATH}:/opt/homebrew/opt/icu4c/lib/pkgconfig:/opt/homebrew/opt/gstreamer/lib/pkgconfig:/opt/homebrew/opt/gst-plugins-base/lib/pkgconfig")
     else()
         add_definitions(-DLINUX)

% cmake -DCMAKE_BUILD_TYPE=Release ../vimix
-- Compiling vimix version 0.7.2
-- GLM: Version 0.9.9.9
-- Compiling 'GLM' OpenGL mathematics https://glm.g-truc.net -- /Users/noah/src/vimix/ext/glm
-- Compiling 'TinyXML2' from https://github.com/leethomason/tinyxml2.git -- /Users/noah/src/vimix/ext/tinyxml2
-- Including 'STB' single-file image and audio processing from https://github.com/nothings/stb -- /Users/noah/src/vimix/ext/stb
-- Compiling 'Ableton Link' from https://github.com/Ableton/link -- /Users/noah/src/vimix/ext/link
-- Compiling 'TinyFileDialog' from https://git.code.sf.net/p/tinyfiledialogs/code -- /Users/noah/src/vimix/ext/tfd.

-----------------------------------------------------------------------------
-- The following external packages were located on your system.
-- This installation will have the extra features provided by these packages.
-----------------------------------------------------------------------------
   * GLib - GTK general-purpose utility library
   * GObject - GTK object-oriented framework
   * GStreamer - Open Source Multiplatform Multimedia Framework
   * GStreamerPluginsBase - GStreamer app library
   * GStreamerPluginsBase - GStreamer audio library
   * GStreamerPluginsBase - GStreamer video library
   * GStreamerPluginsBase - GStreamer pbutils library
   * GStreamerPluginsBase - GStreamer opengl library
   * ICU - International Components for Unicode
   * glfw3 - Open Source multi-platform library for OpenGL

-----------------------------------------------------------------------------
-- Congratulations! All external packages have been found.
-----------------------------------------------------------------------------

-- Including 'GLAD' Open source multi-language OpenGL loader https://glad.dav1d.de -- /Users/noah/src/vimix/ext/glad/include
-- Compiling 'Dear ImGui' from https://github.com/ocornut/imgui.git -- /Users/noah/src/vimix/ext/imgui
-- Compiling 'ImGuiColorTextEdit' from https://github.com/BalazsJako/ImGuiColorTextEdit -- /Users/noah/src/vimix/ext/ImGuiColorTextEdit
-- Compiling 'OSCPack' from http://www.rossbencina.com/code/oscpack -- /Users/noah/src/vimix/ext/OSCPack
-- Using 'CMakeRC ' from https://github.com/vector-of-bool/cmrc.git -- /usr/share/cmake;/Users/noah/src/vimix/cmake/modules.
-- install  gst-plugins /opt/homebrew/Cellar/gstreamer/1.20.4/lib/gstreamer-1.0
-- install  gst-plugins-base /opt/homebrew/lib/gstreamer-1.0
-- install  gst-plugins-bad /opt/homebrew/Cellar/gst-plugins-bad/1.20.3_1/lib/gstreamer-1.0
-- install  gst-plugin-scanner /non_existent_on_purpose/please_use_framework_pkg-config/or_pass_--define-prefix_to_your_pkg-config/libexec/gstreamer-1.0/gst-plugin-scanner
-- install  /opt/homebrew/Cellar/icu4c/71.1/lib/libicuio.dylib;/opt/homebrew/Cellar/icu4c/71.1/lib/libicui18n.dylib;/opt/homebrew/Cellar/icu4c/71.1/lib/libicuuc.dylib;/opt/homebrew/Cellar/icu4c/71.1/lib/libicudata.dylib from /opt/homebrew/Cellar/icu4c/71.1/lib
-- Not signing bundle. Specify APPLE_CODESIGN_IDENTITY to cmake before running cpack to sign
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/noah/src/vimix-build

[brunoherbelin]

wouah! nice job yes agreed: to avoid recompiling gst: it's a beast! BUT: afvideo from OSX is not included... So what I do is compile only gst plug-ins bad from the g streamer installed with brew.

I will send you more info tonight

[nzoschke]

Going to close this in favor of https://github.com/brunoherbelin/vimix/issues/58