Provide an API so that scripts can get user input while running

[stephanelacrampe]

While a script is running, it may require user input, like entering a text, making a choice (Yes/No), or selecting a model element from a list. It would be great if the API would provide a way to do so.

[jamilraichouni]

Oh yes, that would be really awesome. It would also enable to debug scripts with pdb or by embedding an IPython kernel with context to explore any runtime location. Debugging scripts that are run headless via the CLI extension of EASE (e. g. in a Docker container) is crucial. That would also become possible.

I‘m afraid that this is not a trivial one with this Py4J bridging. I already spent a couple of evenings to get user input read in when I run a Python script in an EASE context. Even just in the Eclipse (PyDev) perspective (Eclipse console view) I had no luck with that and really hope to see that somebody knows how to do that!

So far the only hacky thing I had success with was to inject an IPython kernel using gdb (GNU Debugger) in a Linux container. That way I was able to connect a Jupyter console client and land at a dedicated breakpoint position with all the context variables etc. loaded in the given EASE run.

[ylussaud]

For the GUI part you ca check Building UIs with EASE. For an headless mode, depending on how you launch the container you can use the shell itself to prompt the user.

The last point with Jupyter is interesting to us. I tried to make Python4Capella work Jupyter Notebook using the following project: https://github.com/kmhsonnenkind/ease_jupyter_kernels

But I was not able to make it work.

[jamilraichouni]

For an headless mode, depending on how you launch the container you can use the shell itself to prompt the user.

When I put the following two scripts into one and the same folder:

debug.py

breakpoint()

and

debug.sh

#!/bin/bash
CAPELLA_EXE=/Applications/Capella_6.0.app/Contents/MacOS/capella
APP=org.eclipse.ease.runScript

$CAPELLA_EXE -consolelog -application $APP -script "file:$(realpath ./debug.py)"

I see pdb in my terminal but it does not accept/ process any input:

/Applications/Capella_6.0.app/Contents/Eclipse/plugins/org.eclipse.ease.lang.python.py4j_0.8.0.I202104091518/pysrc/ease_py4j_main.py:414: DeprecationWarn
ing: setDaemon() is deprecated, set the daemon attribute instead
  thread.setDaemon(True)
--Return--
> <...>(1)<module>()->None
(Pdb) help

Here, I demonstrate that I entered the command help to get the help of Python's debugger pdb. When I hit Enter nothing happens.

When I call the same script without Capella just directly using Python I see:

$ python3 ./debug.py
--Return--
> /Users/jamilraichouni/dev/novcs/breakpoint_in_ease/debug.py(1)<module>()->None
-> breakpoint()
(Pdb) help

Documented commands (type help <topic>):
========================================
EOF    c          d        h         list      q        rv       undisplay
a      cl         debug    help      ll        quit     s        unt      
alias  clear      disable  ignore    longlist  r        source   until    
args   commands   display  interact  n         restart  step     up       
b      condition  down     j         next      return   tbreak   w        
break  cont       enable   jump      p         retval   u        whatis   
bt     continue   exit     l         pp        run      unalias  where    

Miscellaneous help topics:
==========================
exec  pdb

(Pdb) 

Can you replay that and confirm that Eclipse/ Capella/ EASE really processes the user input and forwards it to pdb, please?

The last point with Jupyter is interesting to us. I tried to make Python4Capella work Jupyter Notebook using the following project

Will look at and reply to that to provide more details in a separate message later (latest by tomorrow).

[jamilraichouni]

Hi @ylussaud!

Could you quickly (max 5 minutes) try this tiny case? I'm really curious to know if stdin is being processed.

By the way:

You can also put the following code into the file debug.py:

print(input("Enter something to be printed: "))

You'll be able to enter a string but I 'm afraid that nothing will be printed (stdin is not processed). This is at least what I see.

[ylussaud]

I have the same issue. I first though it was the Eclipse console that prevent reading from the input, but using the command line I get the same behavior. I guess it has something to do with EASE.

[jamilraichouni]

Okay, thanks for trying and confirming that. Regarding the injection (via GDB) of an IPython kernel into a halted (breakpoint) EASE script: What's the best location to describe that? It is probably a bit unrelated to this issue and it is a bit more complex. I suggest that I open a little dedicated discussion here: https://github.com/labs4capella/python4capella/discussions

[jamilraichouni]

Hi!

Sorry, it took a bit longer to prepare the promised example. Instead of just a couple of words copied from my notes I decided to prepare a demo project here: https://github.com/jamilraichouni/ease-ipython

I hope Docker is fine. The project shows how one can run a Python script with Eclipse EASE and inject an IPython kernel into the running process. This happens with the help of the GNU Debugger (GDB) that attaches to the running Python process. GDB is used to inject the IPython.embed_kernel() command that gives us the full scope of the EASE context.

We can then connect a Jupyter console client and execute stuff like:

from eclipse.system.resources import getWorkspace

getWorkspace().getLocation().toString()

My example uses Eclipse and basic EASE instead of Capella with python4capella just to keep the demo more generic. It works perfectly fine with Capella/ python4capella.

[ylussaud]

Thank you for your explanations and the demonstration repository.

[ylussaud]

As a workaround you can check the tips and trick for user inputs.

[jamilraichouni]

Thanks and at the same time this will probably not help with debugging using something like PDB. By the way. There is a typo in the code on that page: I guess it is supposed to be ComponentDialog instead of ComopnentDialog:

[erik-sjoberg]

I also ran head-first into the issue of having no obvious way to debug python4capella scripts, but I managed to get a relatively simple solution working similar to @jamilraichouni 's approach.

I'm accustomed to using IPython.embed() as a debugging and exploration tool for scripts, but due to the issues with stdin/out this doesn't work within eclipse as mentioned above. Inserting IPython.embed_kernel() would solve this issue, however due to various complicated side-effects of this jupyter kernel it doesn't work in this context.

A simple workaround I found is to use https://pypi.org/project/background-zmq-ipython/ (installable via pip install background_zmq_ipython) to embed an asynchronus ipython kernel without a lot of the complexity of jupyter.

Using the following little helper function, I can insert embed_kernel(user_ns={**globals(), **locals()}) into my script anywhere I'd like to have a breakpoint, and then access the kernel by running jupyter console --existing in a separate terminal on the same machine. This seems to work well and gives me tab-completion of the API and all local variables.

# Include globals and locals in embedded kernel namespace, giving precedence to locals
def embed_kernel(user_ns={**globals(), **locals()}, block=True):
    import time
    import background_zmq_ipython
    user_ns["end_kernel"] = False
    background_zmq_ipython.init_ipython_kernel(user_ns=user_ns)
    if block:
        while not user_ns["end_kernel"]:
            time.sleep(0.2)

I can resume the script execution by setting end_kernel=True in the separate terminal (though there's probably a more elegant way to do this...)

If you do want an asynchronous kernel (and your script doesn't immediately exit) you can also simply use background_zmq_ipython.init_ipython_kernel(user_ns=locals()) instead of the little helper function above.