Interactivity truncated?

[pgierz]

Hi @vsoch,

thanks for build scif, I've been using it recently for some container applications and I wish I discovered this years ago...

I'm having some trouble getting interactivity to work correctly. For some reason, it seems like stderr is being swallowed? I'm not too sure what the issue is exactly, so I'll try to describe it.

Let's say I have a simple Python program which uses some prompts to ask the user for questions. I use the very nice library questionary for this:

#!/usr/bin/env python3
import questionary
from questionary import Style

custom_style_dope = Style(
    [
        ("separator", "fg:#6C6C6C"),
        ("qmark", "fg:#FF9D00 bold"),
        ("question", ""),
        ("selected", "fg:#5F819D"),
        ("pointer", "fg:#FF9D00 bold"),
        ("answer", "fg:#5F819D bold"),
    ]
)

def main():
    questionary.text("What's your first name").ask()
    questionary.password("What's your secret?").ask()
    questionary.confirm("Are you amazed?").ask()
    questionary.select(
        "What do you want to do?",
        choices=["Order a pizza", "Make a reservation", "Ask for opening hours"],
    ).ask()
    questionary.rawselect(
        "What do you want to do?",
        choices=["Order a pizza", "Make a reservation", "Ask for opening hours"],
    ).ask()
    questionary.checkbox(
        "Select toppings", choices=["foo", "bar", "bazz"], style=custom_style_dope
    ).ask()
    questionary.path("Path to the projects version file").ask()

if __name__ == "__main__":
    main()

Based upon the hello world example, I have this as a scif recipe:

%apprun questionary_demo
    python3 $SCIF_APPBIN/questionary_demo.py $@ 2>&1

%applabels questionary_demo
    MAINTAINER pgierz
    VERSION 1.0
%apphelp questionary_demo
    This is an example "Hello World" application using questionary. You can install it to a
    Scientific Filesystem (scif) with the command:
        scif install questionary_demo.scif
    It takes one argument, it will just print Hello, {name}! with the argument when you
    run it. To do that:
        scif run questionary_demo
    if you need to install scif:
        pip install scif
    That step is already done for you in the container. `scif` is the default
    entry point, so you can just use:
        docker <options> <container_name> run questionary_demo

and finally this as a Dockerfile:

FROM quay.io/scif/scif:latest

# Install questionary
RUN pip install questionary

COPY questionary_demo.scif /tmp/questionary_demo.scif

RUN scif install /tmp/questionary_demo.scif

COPY questionary_demo.py /scif/apps/questionary_demo/bin
RUN chmod +x /scif/apps/questionary_demo/bin/questionary_demo.py

# ENTRYPOINT ["scif"]
ENTRYPOINT ["python", "/scif/apps/questionary_demo/bin/questionary_demo.py"]

I initially thought the problem was in how I was using docker (or, on our HPC, singularity), but the ENTRYPOINT that is active (the lower one) works fine. If I use the container entry point as scif and then do

$ docker run -it questionary_demo:latest run questionary_demo

I get:

docker run -it questionary_demo:latest run questionary_demo
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
[questionary_demo] executing /bin/bash /scif/apps/questionary_demo/scif/runscript
? What's your first name

I can still enter info, but I can't see anything I type.

I thought next: OK, maybe it is the containerised version. The same behaviour happens if I install the questionary demo directly onto a scif running on my laptop.

Any hints about how to solve this would be really great!!

Cheers, Paul

[vsoch]

This is a concerning issue first:

docker run -it questionary_demo:latest run questionary_demo
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

You are trying to shell into an x86 image from arm - I'm surprised it let's you get further than that, period. Try pulling down an image that matches the platform first to get rid of that message.

[pgierz]

I thought this was iffy as well (the new Apple ARM chips make life more difficult than it needs to be), so I re-ran my bug on an Ubuntu 20.04. First, on bare-metal:

$ export SCIF_BASE=${HOME}/.local/scif
$ scif install questionary_demo.scif
$ scif apps
questionary_demo
$ scif run questionary_demo

[questionary_demo] executing /bin/bash /home/csys/pgierz/.local/scif/apps/questionary_demo/scif/runscript
... hangs, interactive prompts vanish into nirvana :-( ...

It does accept input. I can see when I Control-C out.

Then, I tried it in a Docker container. Note the warning about mismatched architecture doesn't occur here:

$ docker run -t interactive_scif:latest run questionary_demo
[questionary_demo] executing /bin/bash /scif/apps/questionary_demo/scif/runscript
? What's your first name

A bit better. The prompt is there, input is echoed (however, after several newlines), but the next prompt never shows up.

If that helps already to debug, great. I'll keep looking into it (maybe reproduce it with pure shell), but maybe you have an idea already?

[vsoch]

For the second - any difference when you add -i for "interactive" ?

$ docker run -it interactive_scif:latest run questionary_demo

[pgierz]

Whoops, yes, sorry that was a typo in my comment. But for completeness, with interactive leads to:

docker run -it interactive_scif:latest run questionary_demo
? What's your first name

                          Paul? What's your first name Paul
? What's your secret?

                       ***? What's your secret? ***

The next prompt appears, but input is not echoed as I type, and I only see the response (together with the prompt) after hitting return.

Maybe a thought: How are these messages generated?

pgierz@rz-006962:~$ docker run -it interactive_scif:latest run questionary_demo
[questionary_demo] executing /bin/bash /scif/apps/questionary_demo/scif/runscript
? What's your first name

In my terminal they pop up in pink and seem to come from here: https://github.com/vsoch/scif/blob/555bb059104d070697e495f5c54b6ac6c745a98c/scif/main/commands.py#L68-L75

If I am seeing it correctly, the bot is this: https://github.com/vsoch/scif/blob/555bb059104d070697e495f5c54b6ac6c745a98c/scif/logger/message.py#L339-L340

Maybe the colorise is messing up with the ending of the string?

[vsoch]

@pgierz try inserting an IPython embed around like 80 (in commands.py) there and walking through the logic. I don't think scif was intended to receive input and that's the issue you are running into. If you have a suggestion for a PR / change I'd be happy to review it!