Run shiny app from Rstudio

[vnijs]

In earlier versions of jupyter-rsession-proxy where Rstudio was accessible from a dropdown it was possible to launch a shiny app or gadget from Rstudio. In the latest version where Rstudio is accessible through launcher icons (very nice!) this no longer works however.

Below screenshots from two images on dockerhub using the "older" and "newer" jupyter-rsession-proxy settings.

Icon:

vnijs/rsm-jupyterhub:1.2.1

Dropdown: vnijs/rsm-jupyterhub:1.1.0

[yuvipanda]

@ryanlovett I wonder if this is one of the reasons we switched to rstudio over rsession? I think the proxying code might be there. We can possibly do the proxying ourselves, but unsure what port the hash is trying to call.

As a 'temporary' fix we could possibly allow a configuration to use rstudio again instead of rsession, but that's not a long term tenable situation.

[yuvipanda]

@vnijs can you try:

def _get_rserver_cmd(port):
    # Other paths rsession maybe in
    other_paths = [
        # When rstudio-server deb is installed
        '/usr/lib/rstudio-server/bin/rserver',
        # When just rstudio deb is installed
        '/usr/lib/rstudio/bin/rserver',
    ]
    if shutil.which('rserver'):
        executable = 'rserver'
    else:
        for op in other_paths:
            if os.path.exists(op):
                executable = op
                break
        else:
            raise FileNotFoundError('Can not find rserver in PATH')

    return [
        executable,
        '--www-port=' + str(port)
    ]

c.ServerProxy.servers = {
    'rstudio-rserver': {
        'command': _get_rserver_cmd,
        'launcher_entry': {
            'title': 'RStudio (via RServer)',
        }
    }
}

And see if that's better?

[vnijs]

After importing shutil that gives a 404 You are requesting a page that does not exist warnings and lots of output shown below.

[D 21:41:28.861 NotebookApp] Bad message (TypeError('not all arguments converted during string formatting',)): {'name': 'NotebookApp', 'msg': 'Trying to start {}', 'args': ('rstudio-rserver',), 'levelname': 'DEBUG', 'levelno': 10, 'pathname': '/usr/local/lib/python3.6/dist-packages/simpervisor/process.py', 'filename': 'process.py', 'module': 'process', 'exc_info': None, 'exc_text': None, 'stack_info': None, 'lineno': 60, 'funcName': '_debug_log', 'created': 1546724488.861613, 'msecs': 861.6130352020264, 'relativeCreated': 81954.43391799927, 'thread': 139947436808000, 'threadName': 'MainThread', 'processName': 'MainProcess', 'process': 9, 'action': 'try-start', 'proccess-name': 'rstudio-rserver', 'process-args': ('rserver', '--www-port=46223'), 'process-kwargs': {'env': {'LC_ALL': 'en_US.UTF-8', 'LD_LIBRARY_PATH': '/usr/lib/R/lib:/lib:/usr/lib/x86_64-linux-gnu:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server', 'SHINY_VERSION': '1.5.9.923', 'RSTUDIO_VERSION': '1.2.1206', 'LANG': 'en_US.UTF-8', 'HOSTNAME': 'fb3f4fab98b4', 'NB_UID': '1002', 'SPARK_VERSION': '2.3.2', 'JPASSWORD': 'jupyter', 'PYBASE': '/home/jovyan/.rsm-msba', 'JUPYTER_RUNTIME_DIR': '/home/jovyan/.rsm-msba/share/jupyter/runtime', 'TINI_VERSION': 'v0.18.0', 'DOCKERHUB_VERSION': '1.2.1', 'PWD': '/home/jovyan', 'APT_KEY_DONT_WARN_ON_DANGEROUS_USAGE': 'DontWarn', 'HOME': '/home/jovyan', 'JUPYTER_PATH': '/home/jovyan/.rsm-msba/share/jupyter', 'DEBIAN_FRONTEND': 'noninteractive', 'SPARK_HOME': '/opt/spark-2.3.2-bin-hadoop2.7', 'NB_USER': '0', 'HADOOP_VERSION': '2.7', 'SHELL': '/bin/bash', 'PYSPARK_DRIVER_PYTHON': 'jupyter', 'SHLVL': '0', 'PYSPARK_DRIVER_PYTHON_OPTS': 'lab', 'NB_GID': '1002', 'PYTHONUSERBASE': '/home/jovyan/.rsm-msba', 'PYSPARK_PYTHON': '/usr/bin/python3', 'PATH': '/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/lib/rstudio-server/bin', 'JUPYTER_CONFIG_DIR': '/home/jovyan/.rsm-msba/jupyter'}}}
[D 21:41:28.875 NotebookApp] Bad message (TypeError('not all arguments converted during string formatting',)): {'name': 'NotebookApp', 'msg': 'Started {}', 'args': ('rstudio-rserver',), 'levelname': 'DEBUG', 'levelno': 10, 'pathname': '/usr/local/lib/python3.6/dist-packages/simpervisor/process.py', 'filename': 'process.py', 'module': 'process', 'exc_info': None, 'exc_text': None, 'stack_info': None, 'lineno': 60, 'funcName': '_debug_log', 'created': 1546724488.8754053, 'msecs': 875.4053115844727, 'relativeCreated': 81968.22619438171, 'thread': 139947436808000, 'threadName': 'MainThread', 'processName': 'MainProcess', 'process': 9, 'action': 'started', 'proccess-name': 'rstudio-rserver', 'process-args': ('rserver', '--www-port=46223'), 'process-kwargs': {'env': {'LC_ALL': 'en_US.UTF-8', 'LD_LIBRARY_PATH': '/usr/lib/R/lib:/lib:/usr/lib/x86_64-linux-gnu:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server', 'SHINY_VERSION': '1.5.9.923', 'RSTUDIO_VERSION': '1.2.1206', 'LANG': 'en_US.UTF-8', 'HOSTNAME': 'fb3f4fab98b4','NB_UID': '1002', 'SPARK_VERSION': '2.3.2', 'JPASSWORD': 'jupyter', 'PYBASE': '/home/jovyan/.rsm-msba', 'JUPYTER_RUNTIME_DIR': '/home/jovyan/.rsm-msba/share/jupyter/runtime', 'TINI_VERSION': 'v0.18.0', 'DOCKERHUB_VERSION': '1.2.1', 'PWD': '/home/jovyan', 'APT_KEY_DONT_WARN_ON_DANGEROUS_USAGE': 'DontWarn', 'HOME': '/home/jovyan', 'JUPYTER_PATH': '/home/jovyan/.rsm-msba/share/jupyter','DEBIAN_FRONTEND': 'noninteractive', 'SPARK_HOME': '/opt/spark-2.3.2-bin-hadoop2.7', 'NB_USER': '0', 'HADOOP_VERSION': '2.7', 'SHELL': '/bin/bash', 'PYSPARK_DRIVER_PYTHON': 'jupyter', 'SHLVL': '0', 'PYSPARK_DRIVER_PYTHON_OPTS': 'lab', 'NB_GID': '1002', 'PYTHONUSERBASE': '/home/jovyan/.rsm-msba', 'PYSPARK_PYTHON': '/usr/bin/python3', 'PATH': '/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/lib/rstudio-server/bin', 'JUPYTER_CONFIG_DIR': '/home/jovyan/.rsm-msba/jupyter'}}}
[D 21:41:28.899 NotebookApp] Connection to http://localhost:46223 refused
[D 21:41:28.900 NotebookApp] Bad message (TypeError('not all arguments converted during string formatting',)): {'name': 'NotebookApp', 'msg': 'Readyness: {} after {} seconds, next check in {}s', 'args': (False, 0.02319049835205078, 0.01), 'levelname': 'DEBUG', 'levelno': 10, 'pathname': '/usr/local/lib/python3.6/dist-packages/simpervisor/process.py', 'filename': 'process.py', 'module': 'process', 'exc_info': None, 'exc_text': None, 'stack_info': None, 'lineno': 60, 'funcName': '_debug_log', 'created': 1546724488.90039, 'msecs': 900.3899097442627, 'relativeCreated': 81993.2107925415, 'thread': 139947436808000, 'threadName': 'MainThread','processName': 'MainProcess', 'process': 9, 'action': 'ready-wait', 'proccess-name': 'rstudio-rserver', 'process-args': ('rserver', '--www-port=46223'), 'process-kwargs': {'env': {'LC_ALL': 'en_US.UTF-8', 'LD_LIBRARY_PATH': '/usr/lib/R/lib:/lib:/usr/lib/x86_64-linux-gnu:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server', 'SHINY_VERSION': '1.5.9.923', 'RSTUDIO_VERSION': '1.2.1206', 'LANG': 'en_US.UTF-8', 'HOSTNAME': 'fb3f4fab98b4', 'NB_UID': '1002', 'SPARK_VERSION': '2.3.2', 'JPASSWORD': 'jupyter', 'PYBASE': '/home/jovyan/.rsm-msba', 'JUPYTER_RUNTIME_DIR': '/home/jovyan/.rsm-msba/share/jupyter/runtime', 'TINI_VERSION': 'v0.18.0', 'DOCKERHUB_VERSION': '1.2.1', 'PWD': '/home/jovyan', 'APT_KEY_DONT_WARN_ON_DANGEROUS_USAGE': 'DontWarn', 'HOME': '/home/jovyan', 'JUPYTER_PATH': '/home/jovyan/.rsm-msba/share/jupyter', 'DEBIAN_FRONTEND': 'noninteractive', 'SPARK_HOME': '/opt/spark-2.3.2-bin-hadoop2.7', 'NB_USER': '0', 'HADOOP_VERSION': '2.7', 'SHELL': '/bin/bash', 'PYSPARK_DRIVER_PYTHON': 'jupyter', 'SHLVL': '0', 'PYSPARK_DRIVER_PYTHON_OPTS': 'lab', 'NB_GID': '1002', 'PYTHONUSERBASE': '/home/jovyan/.rsm-msba', 'PYSPARK_PYTHON': '/usr/bin/python3', 'PATH': '/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/lib/rstudio-server/bin', 'JUPYTER_CONFIG_DIR': '/home/jovyan/.rsm-msba/jupyter'}}, 'wait_time': 0.01, 'ready': False, 'elapsed_time': 0.02319049835205078}
[D 21:41:28.914 NotebookApp] Connection to http://localhost:46223 refused
[D 21:41:28.914 NotebookApp] Bad message (TypeError('not all arguments converted during string formatting',)): {'name': 'NotebookApp', 'msg': 'Readyness: {} after {} seconds, next check in {}s', 'args': (False, 0.037207603454589844, 0.02), 'levelname': 'DEBUG', 'levelno': 10, 'pathname': '/usr/local/lib/python3.6/dist-packages/simpervisor/process.py', 'filename': 'process.py', 'module': 'process', 'exc_info': None, 'exc_text': None, 'stack_info': None, 'lineno': 60, 'funcName': '_debug_log', 'created': 1546724488.9145947, 'msecs': 914.5946502685547, 'relativeCreated': 82007.4155330658, 'thread': 139947436808000, 'threadName': 'MainThread', 'processName': 'MainProcess', 'process': 9, 'action': 'ready-wait', 'proccess-name': 'rstudio-rserver', 'process-args': ('rserver', '--www-port=46223'), 'process-kwargs': {'env': {'LC_ALL': 'en_US.UTF-8', 'LD_LIBRARY_PATH': '/usr/lib/R/lib:/lib:/usr/lib/x86_64-linux-gnu:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server', 'SHINY_VERSION': '1.5.9.923', 'RSTUDIO_VERSION': '1.2.1206', 'LANG': 'en_US.UTF-8', 'HOSTNAME': 'fb3f4fab98b4', 'NB_UID': '1002', 'SPARK_VERSION': '2.3.2', 'JPASSWORD': 'jupyter', 'PYBASE': '/home/jovyan/.rsm-msba', 'JUPYTER_RUNTIME_DIR': '/home/jovyan/.rsm-msba/share/jupyter/runtime', 'TINI_VERSION': 'v0.18.0','DOCKERHUB_VERSION': '1.2.1', 'PWD': '/home/jovyan', 'APT_KEY_DONT_WARN_ON_DANGEROUS_USAGE': 'DontWarn', 'HOME': '/home/jovyan','JUPYTER_PATH': '/home/jovyan/.rsm-msba/share/jupyter', 'DEBIAN_FRONTEND': 'noninteractive', 'SPARK_HOME': '/opt/spark-2.3.2-bin-hadoop2.7', 'NB_USER': '0', 'HADOOP_VERSION': '2.7', 'SHELL': '/bin/bash', 'PYSPARK_DRIVER_PYTHON': 'jupyter', 'SHLVL': '0', 'PYSPARK_DRIVER_PYTHON_OPTS': 'lab', 'NB_GID': '1002', 'PYTHONUSERBASE': '/home/jovyan/.rsm-msba', 'PYSPARK_PYTHON': '/usr/bin/python3', 'PATH': '/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/lib/rstudio-server/bin', 'JUPYTER_CONFIG_DIR': '/home/jovyan/.rsm-msba/jupyter'}}, 'wait_time': 0.02, 'ready': False, 'elapsed_time': 0.037207603454589844}
[D 21:41:29.362 NotebookApp] Got code 200 back from http://localhost:46223
[D 21:41:29.363 NotebookApp] Bad message (TypeError('not all arguments converted during string formatting',)): {'name': 'NotebookApp', 'msg': 'Readyness: {} after {} seconds, next check in {}s', 'args': (True, 0.4859185218811035, 0.04), 'levelname': 'DEBUG', 'levelno': 10, 'pathname': '/usr/local/lib/python3.6/dist-packages/simpervisor/process.py', 'filename': 'process.py', 'module':'process', 'exc_info': None, 'exc_text': None, 'stack_info': None, 'lineno': 60, 'funcName': '_debug_log', 'created': 1546724489.3630655, 'msecs': 363.0654811859131, 'relativeCreated': 82455.88636398315, 'thread': 139947436808000, 'threadName': 'MainThread', 'processName': 'MainProcess', 'process': 9, 'action': 'ready-wait', 'proccess-name': 'rstudio-rserver', 'process-args': ('rserver', '--www-port=46223'), 'process-kwargs': {'env': {'LC_ALL': 'en_US.UTF-8', 'LD_LIBRARY_PATH': '/usr/lib/R/lib:/lib:/usr/lib/x86_64-linux-gnu:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server', 'SHINY_VERSION': '1.5.9.923', 'RSTUDIO_VERSION': '1.2.1206', 'LANG': 'en_US.UTF-8', 'HOSTNAME': 'fb3f4fab98b4', 'NB_UID': '1002', 'SPARK_VERSION': '2.3.2', 'JPASSWORD': 'jupyter', 'PYBASE': '/home/jovyan/.rsm-msba', 'JUPYTER_RUNTIME_DIR': '/home/jovyan/.rsm-msba/share/jupyter/runtime', 'TINI_VERSION': 'v0.18.0', 'DOCKERHUB_VERSION': '1.2.1', 'PWD': '/home/jovyan', 'APT_KEY_DONT_WARN_ON_DANGEROUS_USAGE': 'DontWarn', 'HOME': '/home/jovyan', 'JUPYTER_PATH': '/home/jovyan/.rsm-msba/share/jupyter', 'DEBIAN_FRONTEND': 'noninteractive', 'SPARK_HOME': '/opt/spark-2.3.2-bin-hadoop2.7', 'NB_USER': '0', 'HADOOP_VERSION': '2.7', 'SHELL': '/bin/bash', 'PYSPARK_DRIVER_PYTHON': 'jupyter', 'SHLVL': '0', 'PYSPARK_DRIVER_PYTHON_OPTS': 'lab', 'NB_GID': '1002', 'PYTHONUSERBASE': '/home/jovyan/.rsm-msba', 'PYSPARK_PYTHON': '/usr/bin/python3', 'PATH': '/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/lib/rstudio-server/bin', 'JUPYTER_CONFIG_DIR': '/home/jovyan/.rsm-msba/jupyter'}}, 'wait_time': 0.04, 'ready': True, 'elapsed_time': 0.4859185218811035}
[I 21:41:29.368 NotebookApp] 302 GET /rstudio-rserver/ (172.17.0.1) 508.86ms
[D 21:41:29.377 NotebookApp] Using contents: services/contents
[W 21:41:29.382 NotebookApp] 404 GET /auth-sign-in (172.17.0.1) 7.16ms referer=http://127.0.0.1:8888/tree
[D 21:41:29.401 NotebookApp] 304 GET /custom/custom.css (172.17.0.1) 4.39ms

[ryanlovett]

@ryanlovett I wonder if this is one of the reasons we switched to rstudio over rsession? I think the proxying code might be there. We can possibly do the proxying ourselves, but unsure what port the hash is trying to call.

I don't think it was one of the reasons for the switch, but I suppose that rstudio could have been doing some proxying. I'll try out the images in https://hub.docker.com/r/vnijs/rsm-jupyterhub/ to see how this works.

[vnijs]

Thanks @ryanlovett. If you need me to test or change something please let me know. FYI I use the following script to launch the container.

https://github.com/radiant-rstats/docker/blob/master/launch-rsm-jupyterhub.sh

The following will also work:

docker run --rm -p 8888:8888 -e NB_USER=0 -e NB_UID=1002 -e NB_GID=1002 -v ~:/home/jovyan vnijs/rsm-jupyterhub:latest

[vnijs]

@ryanlovett and @yuvipanda

I expect the following is a related issue. Running the usethis code below in rstudio server (e.g., from the vnijs/rsm-msba:latest image on port 8787) restarts Rstudio and loads a project. This also worked as expected with the "old" jupyter-rsession-proxy (e.g., vnijs/rsm-jupyterlab:1.0.0). However, now when RStudio is opened through JupyterLab the only thing shown after following the usethis prompts is "Stream closed" (vnijs/rsm-jupyterlab:latest)

usethis::use_course("https://www.dropbox.com/sh/t77a8d5jqdbw788/AACjng6YJHJV9Rn8nWasOC6Ya?dl=1")

[vnijs]

@ryanlovett and @yuvipanda: It seems like this might be tricky to resolve in the short term. We are hoping to use rsession-proxy on a server for our students that will (hopefully) come online next week. Could you perhaps recommend how to best revert to the previous rsession-proxy setup for now? Below is what I currently use in the Docker file. Thanks

RUN pip3 install jupyter-rsession-proxy \
  && jupyter labextension install jupyterlab-server-proxy

[yuvipanda]

@vnijs to completely use the previous setup, you can the last released version of nbrsessionproxy - https://github.com/jupyterhub/jupyter-rsession-proxy/tree/v0.8.0

I'll take a look at this later today though, I think the fix to moving back to rserver shouldn't be too hard.

Thank you very much for testing and reporting issues!

[vnijs]

That sounds great @yuvipanda!

FYI I tried the setup below based on the link you provided but do not see an option to start Rstudio.

RUN git clone https://github.com/jupyterhub/jupyter-rsession-proxy \
  && cd jupyter-rsession-proxy \
  && git checkout 66485d0276fbe4928758bfd71596f01ba528a62c \
  && pip3 install . \
  && jupyter labextension install jupyterlab-server-proxy \
  && cd - \
  && rm -rf jupyter-rsession-proxy

jovyan@mbp ~ jupyter labextension list

JupyterLab v0.35.4
Known labextensions:
   app dir: /usr/local/share/jupyter/lab
        @jupyter-widgets/jupyterlab-manager v0.38.1  enabled  OK
        @jupyterlab/git v0.5.0  enabled  OK
        @ryantam626/jupyterlab_code_formatter v0.1.5  enabled  OK
        jupyterlab-server-proxy v0.1.0-beta2  enabled  OK

[yuvipanda]

@vnijs yeah, with the old setup there's no explicit icon - you need to go to /rstudio as a URL (rather than /lab)

[yuvipanda]

I'm playing with the following config:

import getpass
import shutil
import os

def _get_rserver_cmd(port):
    # Other paths rsession maybe in
    other_paths = [
        # When rstudio-server deb is installed
        '/usr/lib/rstudio-server/bin/rserver',
        # When just rstudio deb is installed
        '/usr/lib/rstudio/bin/rserver',
    ]
    if shutil.which('rserver'):
        executable = 'rserver'
    else:
        for op in other_paths:
            if os.path.exists(op):
                executable = op
                break
        else:
            raise FileNotFoundError('Can not find rserver in PATH')

    return [
        executable,
        '--www-port=' + str(port)
    ]

c.ServerProxy.servers = {
    'rstudio-rserver': {
        'command': _get_rserver_cmd,
        'environment': {
            'USER': getpass.getuser()
        },
        'launcher_entry': {
            'title': 'RStudio (via RServer)',
        }
    }
}

and it seems to work for me. @vnijs can you give that a shot?

[yuvipanda]

(with new jupyter-server-proxy - I just released version 1.0.0beta7)

[vnijs]

Excellent! Shiny apps work as expected when launched from "Rstudio (via Rserver)".

[vnijs]

The new server for our students has arrived and we are almost ready to make it available to them. The final thing I need is to de-activate the rsession based icon for rstudio and have only the rserver based icon shown. Is there a way to do that?

[yuvipanda]

@vnijs I made RStudio (based on RServer) the default in https://github.com/jupyterhub/jupyter-rsession-proxy/pull/57, and also pushed a new release of jupyter-rsession-proxy (1.0.0beta6) out just now. So if you install the new version, and remove your local traitlet entry, that should work.

Let me know if that works.

[vnijs]

All shortcuts seem to be working great! Trailet to a specific shiny app in the container (radiant), Shiny apps and gadgets launched from Rstudio, and shiny apps and Rmarkdown files can be run on shiny server by browsing directories.

Thanks @yuvipanda!