Cannot capture some console output in Windows

[hyee]

Hello, I'm experiencing a problem of capturing output via NuProcess, it works well for cmd.exe, but when launching some native CLI consoles such as mysql.exe or psql.exe, the program keeps hanging and cannot capture any output. Below test code for your reference:

import com.zaxxer.nuprocess.NuAbstractProcessHandler;
import com.zaxxer.nuprocess.NuProcess;
import com.zaxxer.nuprocess.NuProcessBuilder;

import java.nio.ByteBuffer;
import java.util.Arrays;
import java.util.LinkedList;
import java.util.concurrent.Semaphore;

/**
 * Created by Will on 2017/1/15.
 */
public class MySQLTest {
    public static void main(String... args) {
        new MySQLTest().execute();
    }

    private void execute() {
        NuProcessBuilder pb = new NuProcessBuilder(Arrays.asList("D:\\mysql\\bin\\mysql.exe", "--host=127.0.0.1", "--user=root", "--port=3311"));
        ProcessHandler processHandler = new ProcessHandler();
        pb.setProcessListener(processHandler);
        NuProcess np = pb.start();

        processHandler.awaitDisconnection();
    }

    class ProcessHandler extends NuAbstractProcessHandler {
        private NuProcess nuProcess;
        private LinkedList<String> cmdList = new LinkedList<String>();
        private Semaphore disconnected = new Semaphore(0);

        @Override
        public void onStart(NuProcess nuProcess) {
            this.nuProcess = nuProcess;
        }

        public void awaitDisconnection() {
            disconnected.acquireUninterruptibly(1);
        }

        public void clear() {
            cmdList.clear();
        }

        public void close() {
            nuProcess.destroy(false);
        }

        //Send key event
        public void sendKey(String ch) {
            clear();
            nuProcess.writeStdin(ByteBuffer.wrap(ch.getBytes()));
        }

        //Send command
        public void write(String stack) {
            cmdList.add(stack + "\n");
            //nuProcess.hasPendingWrites();
            nuProcess.wantWrite();
        }

        public synchronized Boolean isPending() {
            return nuProcess.hasPendingWrites();
        }

        @Override
        public boolean onStdinReady(ByteBuffer buffer) {
            if (!cmdList.isEmpty()) {
                String cmd = cmdList.poll();
                buffer.put(cmd.getBytes());
                buffer.flip();
            }
            return !cmdList.isEmpty();
        }

        @Override
        public void onStdout(ByteBuffer buffer, boolean closed) {
            int remaining = buffer.remaining();
            byte[] bytes = new byte[remaining];
            buffer.get(bytes);
            System.out.println("Receiving output ...");
            System.out.println(new String(bytes));
            System.out.flush();
            if (closed) {
                disconnected.release();
            }

            // nuProcess.wantWrite();
            // We're done, so closing STDIN will cause the "cat" process to exit
            //nuProcess.closeStdin(true);
        }

        @Override
        public void onStderr(ByteBuffer buffer, boolean closed) {
            this.onStdout(buffer, false);
        }
    }
}

[brettwooldridge]

Some Windows executables require that they be invoked within a native shell. Have you tried invoking mysql.exe, for example, through cmd.exe /c?

[hyee]

Yes I did try using cmd /c to launch mysql.exe, and it didn't work. But if I execute "mysql.exe --help" with NuProcess which is only to print the help message, I can see the output. It seems that mysql.exe uses another output handler after it logins. I think the command and the arguments are correct, and have tested in native console.

[brettwooldridge]

@hyee This works:

package com.zaxxer.nuprocess.example;

import com.zaxxer.nuprocess.NuAbstractProcessHandler;
import com.zaxxer.nuprocess.NuProcess;
import com.zaxxer.nuprocess.NuProcessBuilder;

import java.nio.ByteBuffer;
import java.util.Arrays;
import java.util.LinkedList;
import java.util.concurrent.TimeUnit;

/**
 * Created by Will on 2017/1/15.
 */
public class MySQLTest {
    public static void main(String... args) throws InterruptedException {
        new MySQLTest().execute();
    }

    private void execute() throws InterruptedException {
        NuProcessBuilder pb = new NuProcessBuilder(Arrays.asList("C:\\mysql\\current\\bin\\mysql.exe", "-n", "--host=127.0.0.1", "--user=root", "--port=3306"));
        ProcessHandler processHandler = new ProcessHandler();
        pb.setProcessListener(processHandler);
        NuProcess np = pb.start();

        processHandler.write("use mysql");
        processHandler.write("show tables;");
        processHandler.write("quit");

        np.waitFor(0, TimeUnit.SECONDS);
    }

    class ProcessHandler extends NuAbstractProcessHandler {
        private NuProcess nuProcess;
        private LinkedList<String> cmdList = new LinkedList<String>();

        @Override
        public void onStart(NuProcess nuProcess) {
            this.nuProcess = nuProcess;
        }

        @Override
        public void onExit(int statusCode)
        {
            System.out.printf("Process exited with status code %d.\n", statusCode);
            System.out.flush();
        }

        public void clear() {
            cmdList.clear();
        }

        //Send key event
        public void sendKey(String ch) {
            clear();
            nuProcess.writeStdin(ByteBuffer.wrap(ch.getBytes()));
        }

        //Send command
        public void write(String stack) {
            cmdList.add(stack + "\n");
            nuProcess.wantWrite();
        }

        public synchronized Boolean isPending() {
            return nuProcess.hasPendingWrites();
        }

        @Override
        public boolean onStdinReady(ByteBuffer buffer) {
            if (!cmdList.isEmpty()) {
                String cmd = cmdList.poll();
                buffer.put(cmd.getBytes());
                buffer.flip();
            }
            return !cmdList.isEmpty();
        }

        @Override
        public void onStdout(ByteBuffer buffer, boolean closed) {
            if (!closed) {
                int remaining = buffer.remaining();
                byte[] bytes = new byte[remaining];
                buffer.get(bytes);
                System.out.println("Receiving output ...");
                System.out.println(new String(bytes));
                System.out.flush();
            }
        }

        @Override
        public void onStderr(ByteBuffer buffer, boolean closed) {
            this.onStdout(buffer, false);
        }
    }
}

Note that ProcessHandler.write() does not add much value, except adding a \n. Calling NuProcess.writeStdin() is going to queue up commands and write them in order anyway, without you needing to maintain a LinkedList<String> yourself.

[brettwooldridge]

@hyee The key piece was the "-n" option to mysql (do not buffer output).

[hyee]

@brettwooldridge I tested your code, the output is very different from the behaviour of native mysql console, no prompt, no version information, no seperators between output query fields, many information are expected to be captured but it doesn't. You can try select * from db; or a simply select for a test.

Yes I did test the -n option before, it doesn't work as expected.

[brettwooldridge]

That is not caused by NuProcess, that is caused by mysql 's behavior. It detects when it us truly connected to an interactive session (or not) and changes behavior accordingly.

This is well mentioned all over the internet with respect to piping commands into mysql and piping output from mysql into another program.

If mysql outputs the prompt in non-interactive mode, the piped output would be useless. For example:

$ mysql your_database --password=foo < my_requests.sql | sed 's/\t/,/g' > out.csv

mysql has various output modifiers such as --batch or --xml or --raw etc.

Please do some of your own research, this issue is closed.

[hyee]

Understood, thanks a lot for your help.