Matlab output is not written in real time

[apommel]

To resolve #5, stdout has been redirected to a StringIO. This allows to resolve the encoding problem, but because of this the output is only written to the terminal when the call is finished. This is especially visible and problematic for files with long execution times and output.

The Matlab Engine API seems to block any kind of redirect to anything other than a StringIO, which makes it hard to solve. Even by executing the Matlab call asynchronously, the StringIO does not seem to be readable in "real-time".

[apommel]

After some research, it seems like that as soon as a custom stdout is specified, the output will only be available when the command is done executing. I understand why it works like that, as otherwise the stdout comes actually directly from Matlab and not from the API. The only real fix would be for Matlab to communicate in unicode.

As a workaround, I implemented an option that allows to choose between displaying unicode characters and having real time output.

[BaconPancakes]

Could you point me to that option?

[apommel]

@BaconPancakes It is the option "Unicode Switch" which allows to display Unicode characters. It is off by default, and in this case the output is in real time in the terminal.

[ghost]

https://stackoverflow.com/questions/18421757/live-output-from-subprocess-command

Does this technique work?

[apommel]

It won't work, because there is no way through the Matlab API to get the output in real-time when having a custom stdout. Matlab would have to change how the engine works to fix this. I even tried to get the output as byte from the standard stdout and decode it by another way, but the information was lost before being passed to stdout so there was also no way.

[ghost]

Damn. Are Mathworks aware of it? Has it been reported?

[apommel]

I have seen the topic a few times on their forum. I don't think it is a priority at all though for them, this is not the kind of use they made the engine for. The engine in itself does not seem to be their priority anyway as it can be quite buggy sometimes (there can be an error in the Matlab side for example, but is never returned to Python so the engine hangs forever)

[ghost]

Fwiw I also tried to change the encoding of characters in the English MATLAB locale:

https://uk.mathworks.com/matlabcentral/answers/280988-how-do-i-get-my-matlab-editor-to-read-utf-8-characters-utf-8-characters-in-blank-squares-in-editors#answer_239883

Doesn't affect the terminal because it's buggy and not strongly supported as you said.

[ghost]

I contacted the Mathworks team and got a solution!

"To enable wide-unicode support for Python 2.7 on Linux, configure the build with the --enable-unicode=ucs4 option."

- https://www.mathworks.com/matlabcentral/answers/426193-why-do-i-receive-an-error-about-undefined-symbol-pyunicodeucs4_decodeutf16-when-importing-matlab

[apommel]

I am not sure if it is exactly the same issue as in our case the engine imports correctly. Anyway this is only for Linux and Python 2.7 so definitely not relevant.

[ghost]

Passed that information on, they should give the engine dev time now!

[apommel]

Nice to hear!

[first-dev]

@BaconPancakes It is the option "Unicode Switch" which allows to display Unicode characters. It is off by default, and in this case the output is in real time in the terminal.

the terminal wasn't showing any output because the option "Unicode Switch" was actually on by default it's better to turn it off and add a note about it just to avoid confusion

[havedifficultyinfindingnames]

I'm trying to start the mlterminal.py file in unicode with python2.7, and it runs. However I cannot receive any result whatever command I give. The same py script in standard folder works probably well except for the wrong coding problem. I'm using Chinese for my system and matlab, and the matlab version is R2020b.