Stuck at ExportByteArray

[nikhalster]

session=WolframLanguageSession()
img = Image.open("path_to_image")
data = session.evaluate(wl.CurvatureFlowFilter(img, 50))
z = session.evaluate(wl.ExportByteArray(data, "PNG"))

Changing CurvatureFlowFilter to other functions like 'Binarize' gives the desired output but gets stuck at ExportByteArray.

This is the Trace when process is manually interrupted:

File "/home/nikhal/.PyCharmCE2019.1/config/scratches/scratch_82.py", line 25, in Binarize z = session.evaluate(wl.ExportByteArray(data, "PNG")) File "/usr/local/lib/python3.6/dist-packages/wolframclient/evaluation/kernel/localsession.py", line 259, in evaluate result = self.evaluate_wrap(expr, kwargs) File "/usr/local/lib/python3.6/dist-packages/wolframclient/evaluation/kernel/localsession.py", line 256, in evaluate_wrap return self.evaluate_wrap_future(expr, kwargs).result() File "/usr/lib/python3.6/concurrent/futures/_base.py", line 427, in result self._condition.wait(timeout) File "/usr/lib/python3.6/threading.py", line 295, in wait waiter.acquire() KeyboardInterrupt

Here is my code: binarize.txt

[riccardodivirgilio]

@nikhalster I think this might be a known issue, if I'm right nothing is really stuck, it's just that if you forget to close the kernel session, the program will never close.

can you try to add session.stop() at the end of your script and see if this fixes your problem?

thanks

[riccardodivirgilio]

I fixed your script to use with in order to automatically open / close the session, let me know if that works

https://gist.github.com/riccardodivirgilio/26f72f0749e726c43fcd0f0f2e567843

[nikhalster]

The first time I ran it, it did work. However subsequent runs has my script stuck again. This is the log:

DEBUG:wolframclient.evaluation.kernel.kernelcontroller:Initializing kernel /usr/local/Wolfram/Mathematica/12.0/Executables/WolframKernel using script: /usr/local/lib/python3.6/dist-packages/wolframclient/evaluation/kernel/initkernel.m
DEBUG:wolframclient.evaluation.kernel.zmqsocket:ZMQ socket bound to tcp://127.0.0.1:42953
DEBUG:wolframclient.evaluation.kernel.zmqsocket:ZMQ socket bound to tcp://127.0.0.1:41231
INFO:wolframclient.evaluation.kernel.kernelcontroller:Kernel writes commands to socket: <Socket: uri=tcp://127.0.0.1:42953>
INFO:wolframclient.evaluation.kernel.kernelcontroller:Kernel receives evaluated expressions from socket: <Socket: uri=tcp://127.0.0.1:41231>
DEBUG:wolframclient.evaluation.kernel.zmqsocket:ZMQ socket bound to tcp://127.0.0.1:46879
INFO:wolframclient.evaluation.kernel.kernelcontroller:Initializing Kernel logger on socket tcp://127.0.0.1:46879
DEBUG:wolframclient.evaluation.kernel.kernelcontroller:Start receiving kernel logger messages.
DEBUG:wolframclient.evaluation.kernel.kernelcontroller:Kernel called using command: /usr/local/Wolfram/Mathematica/12.0/Executables/WolframKernel -noprompt -initfile /usr/local/lib/python3.6/dist-packages/wolframclient/evaluation/kernel/initkernel.m -run ClientLibrary`Private`SlaveKernelPrivateStart["tcp://127.0.0.1:42953", "tcp://127.0.0.1:41231", "tcp://127.0.0.1:46879", 1];.
INFO:wolframclient.evaluation.kernel.kernelcontroller:Kernel process started with PID: 20387
INFO:wolframclient.evaluation.kernel.kernelcontroller:Kernel 20387 is ready. Startup took 0.95 seconds.
DEBUG:wolframclient.evaluation.kernel.kernelcontroller:Expression sent to kernel in 0.000375sec
DEBUG:WolframKernel-<tcp://127.0.0.1:46879>:Evaluating a new expression.
DEBUG:wolframclient.evaluation.kernel.kernelcontroller:Expression received from kernel after 0.062403sec
DEBUG:WolframKernel-<tcp://127.0.0.1:46879>:Done evaluating.
DEBUG:WolframKernel-<tcp://127.0.0.1:46879>:Done responding.

When I try to debug the script, it looks like it gets stuck at the first session.evaluate(). This is the log while debugging:

DEBUG:wolframclient.evaluation.kernel.kernelcontroller:Initializing kernel /usr/local/Wolfram/Mathematica/12.0/Executables/WolframKernel using script: /usr/local/lib/python3.6/dist-packages/wolframclient/evaluation/kernel/initkernel.m
DEBUG:wolframclient.evaluation.kernel.zmqsocket:ZMQ socket bound to tcp://127.0.0.1:34311
DEBUG:wolframclient.evaluation.kernel.zmqsocket:ZMQ socket bound to tcp://127.0.0.1:37115
INFO:wolframclient.evaluation.kernel.kernelcontroller:Kernel writes commands to socket: <Socket: uri=tcp://127.0.0.1:34311>
INFO:wolframclient.evaluation.kernel.kernelcontroller:Kernel receives evaluated expressions from socket: <Socket: uri=tcp://127.0.0.1:37115>
DEBUG:wolframclient.evaluation.kernel.zmqsocket:ZMQ socket bound to tcp://127.0.0.1:45253
INFO:wolframclient.evaluation.kernel.kernelcontroller:Initializing Kernel logger on socket tcp://127.0.0.1:45253
DEBUG:wolframclient.evaluation.kernel.kernelcontroller:Start receiving kernel logger messages.
DEBUG:wolframclient.evaluation.kernel.kernelcontroller:Kernel called using command: /usr/local/Wolfram/Mathematica/12.0/Executables/WolframKernel -noprompt -initfile /usr/local/lib/python3.6/dist-packages/wolframclient/evaluation/kernel/initkernel.m -run ClientLibrary`Private`SlaveKernelPrivateStart["tcp://127.0.0.1:34311", "tcp://127.0.0.1:37115", "tcp://127.0.0.1:45253", 1];.
INFO:wolframclient.evaluation.kernel.kernelcontroller:Kernel process started with PID: 20931
INFO:wolframclient.evaluation.kernel.kernelcontroller:Kernel 20931 is ready. Startup took 0.98 seconds.

[riccardodivirgilio]

Yes, there might be a kernel left open the second time you are running the script. Is the evaluate() call now happening in a with block, like in the script I have sent?

thanks

[nikhalster]

nop. I restarted my computer and tried running it again, however it seems like it's stuck. Although the log says the kernel has done evaluating and responding.

Here is the script. Is it working for you? error.txt

[riccardodivirgilio]

Yes it does work on OSX using V12. Can you please send me your SystemInformation["Small"]?

thanks

[riccardodivirgilio]

does the basic example in the docs work for you? I would like to know if the error you are reporting is related to something happening in the kernel or if something python related. can you execute this code?

from wolframclient.language import wlexpr, wl
from wolframclient.evaluation import WolframLanguageSession

with WolframLanguageSession() as session:
    print(session.evaluate(wlexpr("2+2")))
print("session closed")

[nikhalster]

Here's my System Information:

{"Kernel" -> {"SystemID" -> "Linux-x86-64", 
   "ReleaseID" -> "12.0.0.0 (6206961, 2019040701)", 
   "CreationDate" -> 
    DateObject[{2019, 4, 7, 21, 16, 0}, "Instant", "Gregorian", 5.5]},
 "FrontEnd" -> {"OperatingSystem" -> "Unix", 
   "ReleaseID" -> "12.0.0.0 (6206961, 2019040801)", 
   "CreationDate" -> 
    DateObject[{2019, 4, 8, 8, 56, 57}, "Instant", "Gregorian", 5.5]}}

The above snippet sent by you does work for my system. The script I sent also does work if I comment out the Binarize function call.

[DbxDev]

I suspect that the size of the input image makes a difference here.

I ran your script on a 100kb file (ExampleData[{"TestImage", "House"}]) and it worked. Then I used a bigger one generated with RandomImage[1, {2000, 2000}, ColorSpace -> "RGB"]. This time it took a lot of time to evaluate CurvatureFlowFilter. Log says ~80sec.

To better understand where the bottleneck is, I ran the code in a notebook. With an image of size 200x200 pixels, I already get an evaluation timer of a second:

In[1]:= img = RandomImage[1, {200, 200}, ColorSpace -> "RGB"];

In[2]:= AbsoluteTiming[CurvatureFlowFilter[img, 50];]
Out[2]= {1.14688, Null}

No doubt that it would takes tens of seconds for the bigger picture I mentioned above.

A quick comment, not related, in general one should avoid as much as possible to send data back and forth to the kernel. Remove unnecessary intermediate variables whenever you can.

e.g.: rewriting

    data = session.evaluate(wl.CurvatureFlowFilter(img, 50))
    z = session.evaluate(wl.ExportByteArray(data, "PNG"))

as

z = session.evaluate(
        wl.ExportByteArray(
            wl.CurvatureFlowFilter(img, 50), "PNG"
        ))

reduces the data exchanged to only the resulting byte array. Of course your code is much easier to debug and I understand why you would write it this way until it actually works, so feel free to ignore this.

[DbxDev]

Closing the issue.