search

processing / processing-video

GStreamer-based video library for Processing
274 stars 130 forks source link

UnsatistifedLinkError regarding `arm64e` for video library (because M1 not yet supported) #194

Closed brad-miller closed 1 year ago

brad-miller commented 2 years ago
Firstly thank you and the small team of dedicated volunteers System: MacBook Pro Apple M1 Max MacOS Monterey 12.2 ## Description Processing 4.04b While attempting to use built-in examples :video camera or playmovie console: UnsatisfiedLinkError UnsatisfiedLinkError: /Users/name/Library/Caches/JNA/temp/jna8654886617298368058.tmp: dlopen(/Users/name//Library/Caches/JNA/temp/jna8654886617298368058.tmp, 0x0001): tried: '/Users/name//Library/Caches/JNA/temp/jna8654886617298368058.tmp' (fat file, but missing compatible architecture (have 'i386,x86_64', need 'arm64e')), '/usr/lib/jna8654886617298368058.tmp' (no such file) A library used by this sketch relies on native code that is not available. kind regards b
knupel commented 2 years ago

Are you installing the library video before use it ?

jaegonlee commented 2 years ago

video library is not ready for m1 yet.

benfry commented 2 years ago

The ARM (for M1 and M1 Max) build of Processing isn't quite ready to go yet, and the video library doesn't yet have a version that works natively on those platforms. That's why it's not listed on the download page on the site. In the meantime, use the Intel version with video.

brad-miller commented 2 years ago

Thank for your reply Kind regards Brad

Sent from my iPhone

On 3 Feb 2022, at 5:42 am, Ben Fry @.***> wrote:



The ARM (for M1 and M1 Max) build of Processing isn't quite ready to go yet, and the video library doesn't yet have a version that works natively on those platforms. That's why it's not listed on the download page on the site. In the meantime, use the Intel version with video.

— Reply to this email directly, view it on GitHubhttps://github.com/processing/processing-video/issues/194#issuecomment-1028243597, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AB3THKG476V7R5ZDO24F623UZF3KLANCNFSM5NM232GQ. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. You are receiving this because you authored the thread.Message ID: @.***>

jaegonlee commented 2 years ago

I tested video library with Gstreamer 1.20.0, gst1-java-core-1.4.0, JNA 5.10.0, and it works(not perfect). All movie examples are running ok except Reverse example. Capturing webcam is possible, but only with custom pipeline like this. cam = new Capture(this, 640, 480, "pipeline:avfvideosrc device-index=0 ! video/x-raw, width=640, height=480, framerate=30/1");

Gstreamer 1.20.0 has the problem with loading libMoltenVK.dylib, so I replaced it with this. https://github.com/yusefnapora/m1-multimc-hack/blob/master/lwjglnatives/libMoltenVK.dylib (this problem will be fixed next release of gstreamer)

macOS 12.2 / Processing4.0b5 / macbook pro 16(m1 pro)

brad-miller commented 2 years ago

Firstly thank you for your time, will attempt to replicate….

Kind regards

Sent from my iPhone

On 11 Feb 2022, at 1:16 am, jaegonlee @.***> wrote:



I tested video library with Gstreamer 1.20.0, gst1-java-core-1.4.0, JNA 5.10.0, and it works(not perfect). All movie examples are running ok except Reverse example. Capturing webcam is possible, but only with custom pipeline like this. cam = new Capture(this, 640, 480, "pipeline:avfvideosrc device-index=0 ! video/x-raw, width=640, height=480, framerate=30/1");

Gstreamer 1.20.0 has the problem with loading libMoltenVK.dylib, so I replaced it with this. https://github.com/yusefnapora/m1-multimc-hack/blob/master/lwjglnatives/libMoltenVK.dylib (this problem will be fixed next release of gstreamer)

macOS 12.2 / Processing4.0b5 / macbook pro 16(m1 pro)

— Reply to this email directly, view it on GitHubhttps://github.com/processing/processing-video/issues/194#issuecomment-1034972288, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AB3THKEJJAUDC6NVLE2CIQLU2PCB5ANCNFSM5NM232GQ. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. You are receiving this because you authored the thread.Message ID: @.***>

brad-miller commented 2 years ago

I have replaced dependencies with Gstreamer 1.20.0, gst1-java-core-1.4.0, JNA 5.10.0,

Added ibMoltenVK.dylibhttps://github.com/yusefnapora/m1-multimc-hack/blob/master/lwjglnatives/libMoltenVK.dylib

CustomPipeline example evokes test pattern But with cam = new Capture(this, 640, 480, "pipeline:avfvideosrc device-index=0 ! video/x-raw, width=640, height=480, framerate=30/1”);

Evokes: Processing video library using GStreamer 1.20.0 0:00:00.042298000  9712 0x148fde160 ERROR   GST_PIPELINE gst/parse/grammar.y:851:priv_gst_parse_yyparse: no element "avfvideosrc" 0:00:00.043836000  9712 0x148fde160 ERROR   GST_PIPELINE gst/parse/grammar.y:939:priv_gst_parse_yyparse: link has no source @.*** Feb 11, 2022 8:28:07 PM org.freedesktop.gstreamer.Gst parseLaunch WARNING: no element "avfvideosrc"

Using Started with Capture example With cam = new Capture(this, 640, 480, "pipeline:avfvideosrc device-index=0 ! video/x-raw, width=640, height=480, framerate=30/1”);

Evokes: Processing video library using GStreamer 1.20.0 There are no cameras available for capture.

macOS 12.2.1 / processing4.05b / MacBook Pro 14 (m1 max)

Kind regards On 11 Feb 2022, at 1:15 am, jaegonlee @.**@.>> wrote:

I tested video library with Gstreamer 1.20.0, gst1-java-core-1.4.0, JNA 5.10.0, and it works(not perfect). All movie examples are running ok except Reverse example. Capturing webcam is possible, but only with custom pipeline like this. cam = new Capture(this, 640, 480, "pipeline:avfvideosrc device-index=0 ! video/x-raw, width=640, height=480, framerate=30/1");

Gstreamer 1.20.0 has the problem with loading libMoltenVK.dylib, so I replaced it with this. https://github.com/yusefnapora/m1-multimc-hack/blob/master/lwjglnatives/libMoltenVK.dylib (this problem will be fixed next release of gstreamer)

macOS 12.2 / Processing4.0b5 / macbook pro 16(m1 pro)

— Reply to this email directly, view it on GitHubhttps://github.com/processing/processing-video/issues/194#issuecomment-1034972288, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AB3THKEJJAUDC6NVLE2CIQLU2PCB5ANCNFSM5NM232GQ. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. You are receiving this because you authored the thread.Message ID: @.***>

jaegonlee commented 2 years ago

I think there seems to be a problem with applemedia plugin.

  1. check applemedia plugin file(libgstapplemedia.dylib) is in /Library/Frameworks/GStreamer.framework/lib/gstreamer-1.0 folder.
  2. check the plugin open terminal app cd /Library/Frameworks/GStreamer.framework/Commands ./gst-inspect-1.0 | grep applemedia

you'll see the list.

applemedia:  atdec: AudioToolbox based audio decoder
applemedia:  avfassetsrc: Source and decoder for AVFoundation assets
applemedia:  avfdeviceprovider (GstDeviceProviderFactory)
applemedia:  avfvideosrc: Video Source (AVFoundation)
applemedia:  avsamplebufferlayersink: AV Sample video sink
applemedia:  vtdec: Apple VideoToolbox decoder
applemedia:  vtdec_hw: Apple VideoToolbox decoder (hardware only)
applemedia:  vtenc_h264: H.264 encoder
applemedia:  vtenc_h264_hw: H.264 (HW only) encoder
applemedia:  vtenc_prores: Apple ProRes encoder

  1. if not ./gst-inspect-1.0 -b if you see applemedia in the blacklist, the plugin is disabled for some reasons.(my problem was libMoltenVK.dylib file)

  2. check capture devices ./gst-device-monitor-1.0

Device found:

        name  : FaceTime HD Camera
        class : Video/Source
        caps  : video/x-raw(memory:GLMemory), width=1552, height=1552, format=UYVY, framerate=[ 1/1, 30/1 ], texture-target=rectangle
                video/x-raw(memory:GLMemory), width=1328, height=1760, format=UYVY, framerate=[ 1/1, 30/1 ], texture-target=rectangle
                video/x-raw(memory:GLMemory), width=640, height=480, format=UYVY, framerate=[ 1/1, 30/1 ], texture-target=rectangle
                video/x-raw(memory:GLMemory), width=1760, height=1328, format=UYVY, framerate=[ 1/1, 30/1 ], texture-target=rectangle
...
        properties:
                device.api = avf
                avf.unique_id = xxxxx
                avf.model_id = "FaceTime\ HD\ Camera"
                avf.has_flash = false
                avf.has_torch = false
                avf.manufacturer = "Apple\ Inc."
        gst-launch-1.0 avfvideosrc device-index=1 ! ...
  1. try gst-launch ./gst-launch-1.0 avfvideosrc device-index=0 ! video/x-raw,width=640,height=480,framerate=30/1 ! autovideosink
brad-miller commented 2 years ago

Ok issue was I added OK found my issue

I added the new version of libMoltenVK.dylib to the wrong folder

Was place manually in /Library/Frameworks/GStreamer.framework/lib/gstreamer-1.0 folder Rather replacing old version in /Library/Frameworks/GStreamer.framework/lib/folder

Deleted users/bradmiller/.cashe/gstreamer/registry.arm64bin

Reinitialise call from processing IDE List camera(s)

Thank you for your patience!

Kind regards

@.*** Commands % ./gst-inspect-1.0 -b Blacklisted files:

Total count: 0 blacklisted file @.*** Commands % ./gst-inspect-1.0 | grep applemedia applemedia: atdec: AudioToolbox based audio decoder applemedia: avfassetsrc: Source and decoder for AVFoundation assets applemedia: avfdeviceprovider (GstDeviceProviderFactory) applemedia: avfvideosrc: Video Source (AVFoundation) applemedia: avsamplebufferlayersink: AV Sample video sink applemedia: vtdec: Apple VideoToolbox decoder applemedia: vtdec_hw: Apple VideoToolbox decoder (hardware only) applemedia: vtenc_h264: H.264 encoder applemedia: vtenc_h264_hw: H.264 (HW only) encoder applemedia: vtenc_prores: Apple ProRes encoder

@.*** Commands % ./gst-device-monitor-1.0 Probing devices...

Device found:

name : FaceTime HD Camera class : Video/Source caps : video/x-raw(memory:GLMemory), width=1552, height=1552, format=UYVY, framerate=[ 1/1, 30/1 ], texture-target=rectangle video/x-raw(memory:GLMemory), width=1328, height=1760, format=UYVY, framerate=[ 1/1, 30/1 ], texture-target=rectangle video/x-raw(memory:GLMemory), width=640, height=480, format=UYVY, framerate=[ 1/1, 30/1 ], texture-target=rectangle video/x-raw(memory:GLMemory), width=1760, height=1328, format=UYVY, framerate=[ 1/1, 30/1 ], texture-target=rectangle video/x-raw(memory:GLMemory), width=1080, height=1920, format=UYVY, framerate=[ 1/1, 30/1 ], texture-target=rectangle video/x-raw(memory:GLMemory), width=1280, height=720, format=UYVY, framerate=[ 1/1, 30/1 ], texture-target=rectangle video/x-raw(memory:GLMemory), width=1920, height=1080, format=UYVY, framerate=[ 1/1, 30/1 ], texture-target=rectangle video/x-raw, width=1552, height=1552, format={ (string)UYVY, (string)YUY2, (string)NV12, (string)BGRA }, framerate=[ 1/1, 30/1 ] video/x-raw, width=1328, height=1760, format={ (string)UYVY, (string)YUY2, (string)NV12, (string)BGRA }, framerate=[ 1/1, 30/1 ] video/x-raw, width=640, height=480, format={ (string)UYVY, (string)YUY2, (string)NV12, (string)BGRA }, framerate=[ 1/1, 30/1 ] video/x-raw, width=1760, height=1328, format={ (string)UYVY, (string)YUY2, (string)NV12, (string)BGRA }, framerate=[ 1/1, 30/1 ] video/x-raw, width=1080, height=1920, format={ (string)UYVY, (string)YUY2, (string)NV12, (string)BGRA }, framerate=[ 1/1, 30/1 ] video/x-raw, width=1280, height=720, format={ (string)UYVY, (string)YUY2, (string)NV12, (string)BGRA }, framerate=[ 1/1, 30/1 ] video/x-raw, width=1920, height=1080, format={ (string)UYVY, (string)YUY2, (string)NV12, (string)BGRA }, framerate=[ 1/1, 30/1 ] properties: device.api = avf avf.unique_id = 47B4B64B70674B9CAD2BAE273A71F4B5 avf.model_id = "FaceTime\ HD\ Camera" avf.has_flash = false avf.has_torch = false avf.manufacturer = "Apple\ Inc." gst-launch-1.0 avfvideosrc device-index=0 ! ...

Device found:

name : MacBook Pro Speakers class : Audio/Sink caps : audio/x-raw, format=F32LE, layout=interleaved, rate=48000, channels=2, channel-mask=0x0000000000000003 audio/x-raw, format={ (string)F64LE, (string)F64BE, (string)F32LE, (string)F32BE, (string)S32LE, (string)S32BE, (string)U32LE, (string)U32BE, (string)S24_32LE, (string)S24_32BE, (string)U24_32LE, (string)U24_32BE, (string)S24LE, (string)S24BE, (string)U24LE, (string)U24BE, (string)S20LE, (string)S20BE, (string)U20LE, (string)U20BE, (string)S18LE, (string)S18BE, (string)U18LE, (string)U18BE, (string)S16LE, (string)S16BE, (string)U16LE, (string)U16BE, (string)S8, (string)U8 }, layout=interleaved, rate=[ 1, 2147483647 ], channels=2, channel-mask=0x0000000000000003 audio/x-raw, format={ (string)F64LE, (string)F64BE, (string)F32LE, (string)F32BE, (string)S32LE, (string)S32BE, (string)U32LE, (string)U32BE, (string)S24_32LE, (string)S24_32BE, (string)U24_32LE, (string)U24_32BE, (string)S24LE, (string)S24BE, (string)U24LE, (string)U24BE, (string)S20LE, (string)S20BE, (string)U20LE, (string)U20BE, (string)S18LE, (string)S18BE, (string)U18LE, (string)U18BE, (string)S16LE, (string)S16BE, (string)U16LE, (string)U16BE, (string)S8, (string)U8 }, layout=interleaved, rate=[ 1, 2147483647 ], channels=1 gst-launch-1.0 ... ! osxaudiosink device=44

Device found:

name : MacBook Pro Microphone class : Audio/Source caps : audio/x-raw, format=F32LE, layout=interleaved, rate=48000, channels=1 audio/x-raw, format={ (string)F64LE, (string)F64BE, (string)F32LE, (string)F32BE, (string)S32LE, (string)S32BE, (string)U32LE, (string)U32BE, (string)S24_32LE, (string)S24_32BE, (string)U24_32LE, (string)U24_32BE, (string)S24LE, (string)S24BE, (string)U24LE, (string)U24BE, (string)S20LE, (string)S20BE, (string)U20LE, (string)U20BE, (string)S18LE, (string)S18BE, (string)U18LE, (string)U18BE, (string)S16LE, (string)S16BE, (string)U16LE, (string)U16BE, (string)S8, (string)U8 }, layout=interleaved, rate=48000, channels=2, channel-mask=0x0000000000000003 audio/x-raw, format={ (string)F64LE, (string)F64BE, (string)F32LE, (string)F32BE, (string)S32LE, (string)S32BE, (string)U32LE, (string)U32BE, (string)S24_32LE, (string)S24_32BE, (string)U24_32LE, (string)U24_32BE, (string)S24LE, (string)S24BE, (string)U24LE, (string)U24BE, (string)S20LE, (string)S20BE, (string)U20LE, (string)U20BE, (string)S18LE, (string)S18BE, (string)U18LE, (string)U18BE, (string)S16LE, (string)S16BE, (string)U16LE, (string)U16BE, (string)S8, (string)U8 }, layout=interleaved, rate=48000, channels=1 gst-launch-1.0 osxaudiosrc device=51 ! ...

On 12 Feb 2022, at 1:55 am, jaegonlee @.**@.>> wrote:

I think there seems to be a problem with applemedia plugin.

  1. check applemedia plugin file(libgstapplemedia.dylib) is in /Library/Frameworks/GStreamer.framework/lib/gstreamer-1.0 folder.
  2. check the plugin open terminal app cd /Library/Frameworks/GStreamer.framework/Commands ./gst-inspect-1.0 | grep applemedia

you'll see the list.

applemedia: atdec: AudioToolbox based audio decoder applemedia: avfassetsrc: Source and decoder for AVFoundation assets applemedia: avfdeviceprovider (GstDeviceProviderFactory) applemedia: avfvideosrc: Video Source (AVFoundation) applemedia: avsamplebufferlayersink: AV Sample video sink applemedia: vtdec: Apple VideoToolbox decoder applemedia: vtdec_hw: Apple VideoToolbox decoder (hardware only) applemedia: vtenc_h264: H.264 encoder applemedia: vtenc_h264_hw: H.264 (HW only) encoder applemedia: vtenc_prores: Apple ProRes encoder

  1. if not ./gst-inspect-1.0 -b if you see applemedia in the blacklist, the plugin is disabled for some reasons.

  2. check capture devices ./gst-device-monitor-1.0

Device found:

    name  : FaceTime HD Camera
    class : Video/Source
    caps  : video/x-raw(memory:GLMemory), width=1552, height=1552, format=UYVY, framerate=[ 1/1, 30/1 ], texture-target=rectangle
            video/x-raw(memory:GLMemory), width=1328, height=1760, format=UYVY, framerate=[ 1/1, 30/1 ], texture-target=rectangle
            video/x-raw(memory:GLMemory), width=640, height=480, format=UYVY, framerate=[ 1/1, 30/1 ], texture-target=rectangle
            video/x-raw(memory:GLMemory), width=1760, height=1328, format=UYVY, framerate=[ 1/1, 30/1 ], texture-target=rectangle

... properties: device.api = avf avf.unique_id = xxxxx avf.model_id = "FaceTime\ HD\ Camera" avf.has_flash = false avf.has_torch = false avf.manufacturer = "Apple\ Inc." gst-launch-1.0 avfvideosrc device-index=1 ! ...

— Reply to this email directly, view it on GitHubhttps://github.com/processing/processing-video/issues/194#issuecomment-1036296178, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AB3THKAIJ6G7RA2TIMSXT2DU2UPPLANCNFSM5NM232GQ. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. You are receiving this because you authored the thread.Message ID: @.***>

spuder commented 1 year ago

For anyone else running into error on Processing 4 and wants 2 quick things to try. 

Video Library for Processing 4 does not run on this architecture: macos-aarch64
We could not find a system-wide or bundled installation of GStreamer, but video might still work if GStreamer was placed somewhere else
UnsatisfiedLinkError: Can't load library: /Users/foobar/Library/Caches/JNA/temp/jna6337926620439659438.tmp

Option 1

Download the Intel version of processing. It will be slower than the Apple Silicon version, but may work

Option 2:

Install gstreamer globally

brew install gstreamer gst-plugins-base gst-plugins-good

Verify that /opt/homebrew/bin/ is in your path

echo $PATH

Try running gst commands. 

gst-inspect-1.0

Then restart processing.

Option 3:

Wait for the mac native library of gstreamer to be bundled into processing (this issue).

codeanticode commented 1 year ago

Fixed with releases 2.2 and 2.2.1