Trouble with remote control mode

[mrlimbic]

I've been experimenting with remote control mode but have some strange issues on Mac OS.

When controlled from my test app some commands are just ignored. Particularly text related commands. No onscreen displays appear. No text appears when I send arbitrary text overlay. My app is receiving confirmation responses that make it look like all commands did work.

Also CPU usuage for goes through the roof when controlled from my app so laptop fan goes crazy.

However, none of this happens when using the terminal manually. Text overlays work fine then and performance normal.

Any idea why piping commands streamed from another app might cause strange behaviour?

Is there anything I should avoid doing in my code that may be causing Jadeo to choke?

[x42]

Regarding the text: perhaps this is related with how you start Jadeo.app, related to #31 ie. Jadeo does not find the .ttf font; unless you use the wrapper-script or first change-dir to its application folder (the default is a relative path to the binary - see https://github.com/x42/xjadeo/blob/master/contrib/pkg-osx/wrapper.sh#L4 )

As for the CPU usage, I can only guess: Maybe that the pipe's file-descriptor your app use keeps notifying Jadeo and its select() never sleeps. Perhaps you're using blocking i/o..

[mrlimbic]

About the massive CPU difference.. Looking at the activity monitor it seems "idle wakeups" are huge. What would cause a Jadeo thread to wakeup when I am not sending anything anymore?

Yes I am using blocking I/O to read. But even if I don't read response at all it's still there. I am using apache commons-exec library to call Jadeo (it's a java app).

I'll investigate if there's a non blocking way to do it.

[x42]

I'd expect the relevant xjadeo side code would be:

https://github.com/x42/xjadeo/blob/master/src/xjadeo/xjadeo.c#L185 https://github.com/x42/xjadeo/blob/master/src/xjadeo/xjadeo.c#L210

By default xjadeo sleeps at most 1.0/frame-rate seconds and peridocallly checks if there's some work to be performed. But if there is any incoming remote-command, it immediately handles it.

[mrlimbic]

As far as the font issue, setting the working directory to the Resources directory worked.

[mrlimbic]

I tried using NuProcess library, which doesn't use the built in java process stuff at all and supports non-blocking I/O yet performance was still terrible. Why using remote mode from terminal is 5-10% CPU yet from java 100-150% CPU. It wasn't the idle wakeups causing this either because with NuProcess these were quite low. If I sent you a test case would it help work out what is going on? Remote mode from java is unusable currently (on Mac OS at least).

[mrlimbic]

If there was a way to add osd text via the command line then I could just use that instead of remote mode. I need to label each instance of jadeo somehow because there will be multiple instances of jadeo each with different video versions and offsets. It will be confusing if they are not labelled.

[mrlimbic]

If you are curious what I am doing, it is a video edit comparison and reconform tool for sound editors. So when the picture edit changes the sound editor can alter their project quickly to fit the new edit. This could easily be made to work with Ardour, Mixbus etc too. If you look at the gif below, you'll see that the top line on the graph is the original video, the bottom line the revised video. I want to be able to pop up two video players to check the changes are correct when you click on a change set.

Picture edit comparison based on two XMLs

[mrlimbic]

Does this help? I used the dtrace script called syscallbypid.d

The numbers for Jadeo were insanely bigger (when run from java virtual machine) than any other process on the machine.

  571 Jadeo-bin                ulock_wait                  20594
   571 Jadeo-bin                ulock_wake                  20594
   571 Jadeo-bin                read                      1496393
   571 Jadeo-bin                select                    1496393
   571 Jadeo-bin                psynch_mutexdrop          3279947
   571 Jadeo-bin                psynch_mutexwait          3279947

When run from terminal the numbers way lower and very similar ranges to other process. These two were were the highest in that situation.

   643 Jadeo-bin                psynch_cvbroad              13855
   643 Jadeo-bin                psynch_cvwait               16704

[x42]

it confirms my theory, but it doesn't say why select doesn't wait until input becomes available. read here is reading input from your control application.

Are you comfortable with compiling jadeo.app locally to investigate this?

First step would be xcode-select --install (go get xcode commandline tools).

The following is perhaps easiest to run using new, normal (not admin) user-account. That way any potential user-account customizations (eg. homebrew, mac-ports) won't interfere. Mac makes it easy to create, switch-user..

curl -O https://raw.githubusercontent.com/x42/xjadeo/master/x-osx-buildstack.sh
bash x-osx-buildstack.sh

keep fingers crossed (it downloads dependencies and compiles in ~/src/).

mv /tmp/Jadeo*.dmg ~/Desktop

If that works, it would be easy to add some debug messages in the code to investigate this.

[mrlimbic]

It all seemed to go OK until the last step of creating the DMG. So mv /tmp... didn't work.

DMG MB =  22
created: /var/folders/j_/sbky_3r55jbfwcd19tzjshd80000gr/T/xjadeo.uaIvw3Lh.dmg
Initialized /dev/rdisk2s1 as a 22 MB case-insensitive HFS Plus volume
/Users/bob/src/xj_build/xjadeo/contrib/pkg-osx/dmgbg.png -> /var/folders/j_/sbky_3r55jbfwcd19tzjshd80000gr/T/xjadeoimg.emJeXNcd/.background/dmgbg.png
setting DMG background ...
70:74: execution error: Finder got an error: Can’t get disk "xjadeoimg.emJeXNcd". (-1728)
Failed to set background/arrange icons
"disk2" unmounted.
"disk2" ejected.

[x42]

It all seemed to go OK until the last step of creating the DMG. So mv /tmp... didn't work.

Was the user graphically logged in at the time? Creating a DMG requires a "Finder" Process. Alternatively.. have you been able to get the .app directly ?

[mrlimbic]

Ah no. Only switched user in the terminal. I'll try again. Thanks.

[mrlimbic]

OK. That worked. Have a DMG. What now?

[mrlimbic]

I've come back to this to see if can work it out again.

I've found out that my small java dummy control client is basically being blocked when trying to read the response line after sending a command.

For some reason the first response line when opening the connection is fine (no command sent yet). @800 xjadeo - remote control (type 'help' for info)

But after sending the first command then it blocks. ping (Waiting for response..)

This doesn't happen from terminal. Any idea what the difference might be?

[mrlimbic]

Here is a simple java 8 client that reproduces the CPU > 100% problem on Mac OS.

It takes one argument which is the video file to load.

Unlike my previous comment, this example code does not wait for the response after sending a command. Another thread just gobbles all responses separately and echoes them.

It has a dependency on apache commons exec library..

https://commons.apache.org/proper/commons-exec/

import java.io.BufferedReader;
import java.io.File;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.PipedInputStream;
import java.io.PipedOutputStream;

import org.apache.commons.exec.CommandLine;
import org.apache.commons.exec.DefaultExecuteResultHandler;
import org.apache.commons.exec.DefaultExecutor;
import org.apache.commons.exec.ExecuteException;
import org.apache.commons.exec.ExecuteStreamHandler;
import org.apache.commons.exec.PumpStreamHandler;
import org.apache.commons.exec.ShutdownHookProcessDestroyer;

// Jadeo controller wrapper
public class Jadeo implements Runnable {
    public static String JADEO_MAC_BIN = "/Applications/Jadeo.app/Contents/MacOS/Jadeo-bin";
    public static String JADEO_MAC_DIR = "/Applications/Jadeo.app/Contents/Resources";

    PipedInputStream receive;
    BufferedReader reader;
    PipedOutputStream send;

    DefaultExecutor exec;

    public Jadeo() {
    }

    public boolean open() {
        try {
            CommandLine cmd = new CommandLine(JADEO_MAC_BIN);
            cmd.addArgument("--remote");    
            cmd.addArgument("--no-splash"); 
            cmd.addArgument("--ontop"); 

            send = new PipedOutputStream();
            PipedInputStream in = new PipedInputStream(send);

            receive = new PipedInputStream();
            PipedOutputStream out = new PipedOutputStream(receive);
            reader = new BufferedReader(new InputStreamReader(receive));

            ExecuteStreamHandler streamHandler = new PumpStreamHandler(out, null, in);

            DefaultExecuteResultHandler resultHandler = new DefaultExecuteResultHandler();

            exec = new DefaultExecutor();

            exec.setStreamHandler(streamHandler);       
            exec.setProcessDestroyer(new ShutdownHookProcessDestroyer());

            exec.setWorkingDirectory(new File(JADEO_MAC_DIR));
            exec.execute(cmd, resultHandler);

            // Thread to gobble process output
            Thread gobbler = new Thread(this);
            gobbler.setDaemon(true);
            gobbler.start();
        } catch (ExecuteException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }

        return true;
    }

    public void close() {
        try {
            command("quit");
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    @Override
    public void run() {
        String line = null;

        try {
            while ((line = reader.readLine()) != null) {
                // Echo output
                System.out.println(line);
            }
        } catch (IOException e1) {
            e1.printStackTrace();
        }
    }

    public void command(String command) throws IOException {
        send.write(command.getBytes());
        send.write("\n".getBytes());
        send.flush();
        System.out.println(command);
    }

    public static void main(String[] args) {
        // First argument is video file to load
        String video = args[0];

        try {
            Jadeo jadeo = new Jadeo();
            jadeo.open();
            jadeo.command("ping");
            jadeo.command("osd on");
            jadeo.command("osd mode 2");
            jadeo.command("osd box");
            jadeo.command("osd pos 0 95");
            jadeo.command("osd text some arbitrary text overlay here");
            jadeo.command("load " + video);
            jadeo.command("seek 200");
            jadeo.command("midi driver portmidi");
            jadeo.command("midi connect 0");
            jadeo.command("midi sync 0"); 
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

[mrlimbic]

When attempting to repoduce the problem from another language instead of Java I came across this difference in Python. This is probably easier for you to reproduce.

Why the massive difference in CPU usage like this?

100+% CPU usage like this.. cp = subprocess.run(["/Applications/Jadeo.app/Contents/MacOS/Jadeo", "--remote", "--ontop", "--no-splash"], shell=False)

But only 1% CPU usage like this.. cp = subprocess.run(["/Applications/Jadeo.app/Contents/MacOS/Jadeo", "--remote", "--ontop", "--no-splash"], shell=True)

[x42]

if select() indicates that data can be read from it, xjadeo will attempt to do that immediately and not wait. This is entirely intentional and by design.

Since file-descriptor used to communicate with xjadeo is provided by the controlling app, this is outside xjadeo's control.

---

Could it be that requesting shell=True implies stdin=PIPE, stdout=PIPE and hence there is a valid file-descriptor to poll and read from. While without it, it leads to select returning instantly with an error condition? or perhaps there's a difference regarding setting O_NONBLOCK on the pipe?

xjadeo cannot distinguish between valid, and error activity on the controlling fd. Worst case, if there is an error xjadeo could disable the remote-ctrl input. That's not what you want.

Some printf() debugging as suggested earlier may shed some light here. -- maybe check if select() indeed returns -1 rather than a positive value, or if there's constant valid activity for some reason.

https://github.com/x42/xjadeo/blob/4d2fce8ae95c125050c34ae0903522882727556e/src/xjadeo/xjadeo.c#L210