Newtek NDI

[radekolszak]

Could someone give ffmpeg commad to stream hdmi input in NEWtek ndi format. ffmpeg now supports NDI.

[jcoutch]

@radekolszak - I'm also trying to get this to work on my Zidoo X8. Unfortunately, NDI is not compiled into the standard FFMPEG builds. Also, the NDI SDK for Linux only supports NDI senders (not receivers), so some modifications would be required to disable the receiver functionality in FFMPEG.

[jcoutch]

@radekolszak - Just wanted to post an update. I had some free time today, and managed to get FFmpeg compiled with libnewtek_ndi enabled on Android. I have not had a chance to test whether it actually works, but I pushed my build scripts up to GitHub if you feel up to building/testing it yourself:

https://github.com/jcoutch/FFmpeg-Android

I'm hoping to have time tomorrow to test it out with Instreamer on my X8. If everything looks good, I'll reach out to Newtek to check on the legalities of publishing compiled binaries (since their license isn't fully compatible with the GPL.)

[jcoutch]

Had a chance to test the binaries on my X8. Unfortunately, NewTek's ARM library wasn't compiled using the Android NDK toolchain, and won't work. I contacted them to see if the libraries could be included in a future version of their SDK, but at this point, there's not much we can do but wait.

[jcoutch]

@radekolszak - I managed to get video streaming working directly with Newtek's Android app without the need for Instreamer/FFMPEG. Luckily, the version of Android on the Zidoo boxes has V4L2, and the HDMI device is on /dev/video250 (at least on my X8.) If you root your device and run the following commands in a terminal emulator:

su
cd /dev
mknod video0 c 81 0
chown media video0
chgrp system video0

This will create a video0 device that Android will pick up as the default camera, and so will the NDI app. Before starting the stream, click the gear and disable the microphone.

This isn't a perfect solution yet. As soon as an HDMI source is removed, the video0 device will disappear. It will also disappear after reboot. Also, audio recording does not work (which is why you have to disable it before starting the stream.) I'm thinking it's just a matter of finding the correct device and setting the volume...but I don't have time at the moment to dig in further.

[radekolszak]

Thank you. I 've posted link to your soulution on "streaming idiots" group on facebook. But Newtek NDI app is only capable of 720p streaming?

[pabloko]

@radekolszak @jcoutch sorry for reviving old issue but you could just get the ndi camera app and extract libndi.so for your arch, it has same endpoints as original sdk plus a jni intrface, ive just used it to send the android screen taken with MediaProjection. Also tweaking the ndi camera app to give 1080/720 instead 720/460 is very easy, as you only need to edit com.newtek.ndicamera.StreamingNDI.C0790b/m4055a theres call to android api Camera.getSupportedPreviewSizes that gives an array of sizes the camera can output, they are filtered by the multiplication of width*height: int i = z ? 921600 : 230400; change to int i = z ? 2073600 : 921600; recompile, sign, deploy and you will get 1080 feed.

[ddavidebor]

@pabloko I'm trying to decompile with apktool but i cannot find what you sugested, what tool did you use? David

[wsteelenyc]

@pabloko Do you have a working android studio example of using extract libndi.so ?

[pabloko]

@wsteelenyc for obv reasons cant post that, but heres the spec, on package com.newtek.ndicamera.StreamingNDI;

package com.newtek.ndicamera.StreamingNDI;
...
    public native int getConnection(); //Check if ndi connection is active and transmitting
    public native void initAudioParam(int buf_size, int sample_rate); //Init audio buffers
    public native void initVideoParam(int width, int height, int min_fps, int max_fps); //Init video buffers
    public native boolean ndiInit(String ndi_name); //Init NDI stream
    public native void streamerAudio(byte[] audio_frame); //Feed audio data (defult 16000kHz 2ch stereo)
    public native void streamerVideo(byte[] picture_frame); //Feed video data (bitmap argb)

regards

[wsteelenyc]

@pabloko I think that is helpful. I'm new to working with .so files so I'll give it a shot and see if I get anywhere.

[wsteelenyc]

@pabloko So I brought the .so files into my android studio project to into the appropriate jniLibs directory for the to architectures respectively. Then I created a native class "com.newtek.ndicamera.StreamingNDI" and loaded the library in my MainActivity which has it's own seperate namespace with...

static{
    System.loadLibrary("NDIsendAV");
}
public StreamingNDI NDIClass = new StreamingNDI();

I get the following error when I try to call ndiInit "No implementation found for boolean com.newtek.ndicamera.StreamingNDI.ndiInit(java.lang.String) (tried Java_com_newtek_ndicamera_StreamingNDI_ndiInit and Java_com_newtek_ndicamera_StreamingNDI_ndiInit__Ljava_lang_String_2)"

I'm assuming it's either something simple and has something I am forgetting or namespace related. Does my entire application need to exist in the "com.newtek.ndicamera" namespace? Any suggestions?

[pabloko]

yeah, thats because i forgot to include the class name (NDIstreaming), sorry.

it should be

package com.newtek.ndicamera.StreamingNDI;
public class NDIstreaming {
    public NDIstreaming () {
        System.loadLibrary("NDIsendAV");
    }
    public native int getConnection(); //Check if ndi connection is active and transmitting
    public native void initAudioParam(int buf_size, int sample_rate); //Init audio buffers
    public native void initVideoParam(int width, int height, int min_fps, int max_fps); //Init video buffers
    public native boolean ndiInit(String ndi_name); //Init NDI stream
    public native void streamerAudio(byte[] audio_frame); //Feed audio data (defult 16000kHz 2ch stereo)
    public native void streamerVideo(byte[] picture_frame); //Feed video data (bitmap argb)
}

[wsteelenyc]

@pabloko Ahah! That worked! Now on to figure out how to get it to work with CameraX. Wish me luck!

[wsteelenyc]

@pabloko OK. Some progress in that I am getting no errors and I am able to get bitmaps in realtime from cameraX and convert them to a byte array and not seeing any errors, however I am not seeing the NDI stream show up in my studio monitor and when querying NDIClass.getConnection(); It returns 0 which I assume means there isn't one. This should work without passing audio, right? Is anything out of order causing the NDIStreaming not to initialize properly or am I missing something else from the JNI so? Is there a way to catch any errors from the class? I added Internet and Network Status permissions to my app. Some of my relevant code is below.

I initialized the class like so in my MainApplication... static{ System.loadLibrary("NDIsendAV"); } public NDIstreaming NDIClass = new NDIstreaming(); private int NDIConnection;

Then I start my NDI stream after everything is set up... boolean bNDIinit = NDIClass.ndiInit("nditest"); NDIClass.initVideoParam(textureView.getWidth(), textureView.getHeight(), 25, 60); NDIConnection = NDIClass.getConnection();

Then using cameraX imageAnalysis I do... final Bitmap bitmap = textureView.getBitmap(); if(bitmap==null) return; int size = bitmap.getRowBytes() * bitmap.getHeight(); ByteBuffer byteBuffer = ByteBuffer.allocate(size); bitmap.copyPixelsToBuffer(byteBuffer); //ByteArrayInputStream byteArray = byteBuffer.array(); NDIClass.streamerVideo(byteBuffer.array()); NDIConnection = NDIClass.getConnection();

[meduar]

Hi @wsteelenyc, I see you were able to stream video, but I cant. I followed exactly the same I can't get video, I am not using CAMERAX, just the previous one.

Do I have to use, NDIClass.streamerVideo(byteBuffer.array());

How to pass video?

Thanks,

[gabriel-ar]

Hi @pabloko and @wsteelenyc , thanks for documenting all your progress. I was able to get the video stream working, but I could only reach 1280 x 720 @ 30fps. At that resolution, the streamerVideo() function is taking approx. 25 ms to execute. The maximum that the function can take to stream at 30 fps is 33 ms (1000/30), right now it's almost in the limit. My device is a Samsung Galaxy S10+, and I have activated the performance mode and disabled battery optimization for the app. Any suggestions?