Jupytext now fails to open .Rmd in classic notebook

damianavila commented 1 year ago

Context

Ref: https://2i2c.freshdesk.com/a/tickets/459

There was a preliminary investigation from @yuvipanda and then @damianavila spent a few cycles concurring and pointing to a problem in the image itself because of being able to locally reproduce the issue.

Mathew brought new evidence suggesting it might be something on our side as well (our configuration of the hub).

Proposal

I think we need to do another round of investigation given the latest data provided by Matthew.

Updates and actions

Resolution: jupytext is not compatible with nbclassic yet, PR at https://github.com/jupyter/nbclassic/pull/235
More details at https://github.com/2i2c-org/infrastructure/issues/2302#issuecomment-1473825235 and https://github.com/mwouts/jupytext/issues/1051
Workaround described in https://github.com/mwouts/jupytext/issues/1051#issuecomment-1484177786 and https://github.com/mwouts/jupytext/issues/1051#issuecomment-1484292916

matthew-brett commented 1 year ago

Please let me know anything I can to help. This problem is fairly urgent for me - the experience using the hub is significantly worse now - they have a different interface for the textbook (Lab) and the exercises (Classic), and they have to open the textbook link to get a text representation, and then right-click Open with... Notebook in the Lab sidebar to get the textbook pages as notebooks.

matthew-brett commented 1 year ago

Any progress here - or ETA? Would it be simplest for us to move to a self-hosted hub for now?

damianavila commented 1 year ago

Hi @matthew-brett, we have scoped additional resources for making another round of investigation in the current sprint. We are timeboxing our effort around this one and we will get back to you by the end of the week with the results of our investigation and, hopefully, a resolution.

matthew-brett commented 1 year ago

Great - thanks.

GeorgianaElena commented 1 year ago

@matthew-bret , found a workaround to improve a bit the experience on JupyterLab Expand the second bullet in https://jupytext.readthedocs.io/en/latest/index.html#install to learn more about this config.

You can configure what viewer to be used for a certain file format in jlab, by creating a default_setting_overrides.json file in the ~/.jupyter/labconfig directory with at least the following contents:

{
  "@jupyterlab/docmanager-extension:plugin": {
    "defaultViewers": {
      "markdown": "Jupytext Notebook",
      "r-markdown": "Jupytext Notebook",
    }
  }
}

In https://github.com/mwouts/jupytext/blob/main/binder/labconfig/default_setting_overrides.json there's an extended example for more file formats.

Each user can then run the following command to set Jupytext as the default viewer:

wget https://raw.githubusercontent.com/mwouts/jupytext/main/binder/labconfig/default_setting_overrides.json -P  ~/.jupyter/labconfig/

Alternatively, because you have access to all users' home dirs under allusers, you could copy this config into each of their home dirs.

With this, at least rmd and md files will open with Jupytext by default and users won't have to right click on things.

Update:

Note, that this will only change the behaviour when double clicking on files, links won't work because the jupyterlab version in the list user image is < 3.6. But according to https://github.com/mwouts/jupytext/issues/271#issuecomment-1379065846, if you were to upgrade the jupyterlab version to at least 3.6, linked markdown files will render correctly with this config

GeorgianaElena commented 1 year ago

Other tries and findings after poking around this morning:

1. Checked if the most recent change to the hub image is to blame for the change in behavior ❎ conclusion nothing changed

Details: I've deployed the old hub image in https://github.com/2i2c-org/infrastructure/pull/2136/files on the 2i2c-uk staging hub + the lis user image lisacuk/lishub-base:581a714 and opening .md and .rmd files still were downloaded instead of opened

2. On top of 1, I also reverted the changes in https://github.com/2i2c-org/infrastructure/pull/2210/files ❎ conclusion nothing changed

3. Checked behavior on other 2i2c hubs

The 2i2c staging, which uses the default 2i2c user image behaves the same as the lis one when trying to open md files from the classical notebook interface
The 2i2c-dask staging, which uses the pangeo/pangeo-notebook:2022.06.02 behaves a bit different. Double clicking on a markdown file, doesn't try to download it as it happens on the lis hub and the 2i2c staging one, but instead it opens it in edit mode.

matthew-brett commented 1 year ago

Thanks for this. Can I ask, for 2i2c-dask staging, what is the behavior opening .md or .Rmd files in the classic interface? I had not previously tested the lab behavior, so I don't know at the moment whether 2i2c-dask staging is replicating what I had before this arose.

GeorgianaElena commented 1 year ago

Can I ask, for 2i2c-dask staging, what is the behavior opening .md or .Rmd files in the classic interface?

@matthew-brett, the behaviour described in https://github.com/2i2c-org/infrastructure/issues/2302#issuecomment-1471815284 is actually happening when in the classic interface. All files that do not have a ipynb extention get opened in a plain text editior. The fundamental difference of this user image is that is doesn't have jupytext installed. Though, after temp installing, the same thing happened.

Btw, I have set with the configurator the same pangeo image that the 2i2c-dask-staging is using on the 2i2c-uk-staging one, running at https://staging.uk.2i2c.cloud. I gave you admin access into that one if you want to take a look.

GeorgianaElena commented 1 year ago

I think I have just discovered something else. On different browsers, happen different when double clicking .md files.

On Firefox, they get downloaded, whereas on Chrome they get weirdly rendered as raw text

damianavila commented 1 year ago

Other recent reports in binder as well: https://github.com/mwouts/jupytext/issues/1051

damianavila commented 1 year ago

With this, at least rmd and md files will open with Jupytext by default and users won't have to right click on things.

Wondering if when the user volume is getting mounted, we are losing the config files at ~/.jupyterand this is why the later intervention (copy of the config file) fix the opening issue as per @GeorgianaElena's comment.

Instead of putting the config file in ~/.jupyter (https://github.com/lisds/lisds-images/blob/main/lishub-base/Dockerfile#L47-L48) I would suggest maybe using other jupyter paths, ie. /opt/conda/etc/jupyter:

(base) jovyan@9732c2f4fc5c:~$ jupyter --paths
config:
    /home/jovyan/.jupyter
    /home/jovyan/.local/etc/jupyter
    /opt/conda/etc/jupyter
    /usr/local/etc/jupyter
    /etc/jupyter
data:
    /home/jovyan/.local/share/jupyter
    /opt/conda/share/jupyter
    /usr/local/share/jupyter
    /usr/share/jupyter
runtime:
    /home/jovyan/.local/share/jupyter/runtime

Something along these lines:

# Configure lab to open Jupytext docs in notebook interface.
RUN wget https://raw.githubusercontent.com/mwouts/jupytext/main/binder/labconfig/default_setting_overrides.json \
    -P  /opt/conda/etc/jupyter/labconfig/

GeorgianaElena commented 1 year ago

My conclusions after debugging this

1. JupyterLab

On JupyterLab, the solution described in https://github.com/2i2c-org/infrastructure/issues/2302#issuecomment-1471805215 as a workaround is not actually a workaround, but the solution and how the Jupytext itself documents it to be configured in a lab environment, for newest jupyterlab versions.

So, JupyterLab + Jupytext in the current 2i2c hub environment works as expected if configured per the Jupytext documentation.

2. Classical Notebook interface

My conclusion is that Jupytext is not currently fully compatible with the current 2i2c hub environment, nor with latest mybinder deployment. At least not out of the box, or not without some tweaking in jupytext, or adjacent notebook and nbclassic projects.

For example, the example in the Jupytext's documentation itself doesn't work on the classical notebook interface and manifests the same behaviour. Check it out by expanding the first bullet in https://jupytext.readthedocs.io/en/latest/index.html#install and then click on the mybinder link.

A more in-depth study

What it happens when clicking on an .md file from the classical notebook is that the link that gets generated is set correctly by Jupytext to something like https://staging.2i2c.cloud/user/../notebooks/hello.md, but then it gets a 301 and permanently redirected to https://staging.2i2c.cloud/user/.../files/hello.md, (notice notebooks vs file in the URL) hence the different behaviour in different browsers. Some, render the file in browser, some promt you to download the file.

My best guess is that this is coming from nbclassic's default behaviour that somehow Jupytext is unable to overwrite.

I believe this is also related somehow to the changes that migrating mybinder and the 2i2c hubs to use ServerApp instead of NotebookApp. Probably because these changes are the ones that impose the use nbclassic for the classical notebook interface. Unfotunatelly I wasn't able to 'fix' the issue by just reverting the changes there. Maybe I missed something. But going back to NotebookApp should not be the recommended way forward anyway since everything moves the the ServerAPpps' direction.

What to do next

Given that this incompatibility is already captured in the relevant upstream projects:

https://github.com/mwouts/jupytext/issues/1051
https://github.com/jupyter/nbclassic/issues/157 ,and we also see it for the 2i2c default user image. We should try to provide as much details as possible about what we saw it's happening and/or invest some cycles into upstream work so that those issues to come to a solution.

LIS hub

I believe JupyterLab can be used instead of the classical notebook, with Jupytext, now that we concluded that they work ok toghether on the current hub setup, until the upstream issues get resolved.

matthew-brett commented 1 year ago

Hi,

Do you think it is possible that, for some reason, the latest binder / Jupyterhub images are overwriting or disabling the jupytertext configuration?

Cheers,

Matthew

On Fri, Mar 17, 2023 at 1:13 PM Georgiana @.***> wrote:

My conclusions after debugging this 1. JupyterLab

On JupyterLab, the solution described in #2302 (comment) https://github.com/2i2c-org/infrastructure/issues/2302#issuecomment-1471805215 as a workaround is not actually a workaround, but the solution and how the Jupytext itself documents it to be configured in a lab environment, for newest jupyterlab versions.

So, JupyterLab + Jupytext in the current 2i2c hub environment works as expected if configured per the Jupytext documentation.

Classical Notebook interface

My conclusion is that Jupytext is not currently fully compatible with the current 2i2c hub environment, nor with latest mybinder deployment. At least not out of the box, or not without some tweaking in jupytext, or adjacent notebook and nbclassic projects.

For example, the example in the Jupytext's documentation itself doesn't work on the classical notebook interface and manifests the same behaviour. Check it out by expanding the first bullet in https://jupytext.readthedocs.io/en/latest/index.html#install and then click on the mybinder link. A more in-depth study

What it happens when clicking on an .md file from the classical notebook is that the link that gets generated is set correctly by Jupytext to something like https://staging.2i2c.cloud/user/../notebooks/hello.md https://staging.2i2c.cloud/user/../notebooks/hello.md, but then it gets a 301 and permanently redirected to https://staging.2i2c.cloud/user/.../files/hello.md https://staging.2i2c.cloud/user/.../files/hello.md, (notice notebooks vs file in the URL) hence the different behaviour in different browsers. Some, render the file in browser, some promt you to download the file.

My best guess is that this is coming from nbclassic's default behaviour https://github.com/jupyter/nbclassic/blob/f237e24acf1f6a7f08c0593a75a41f5e618b2832/nbclassic/notebookapp.py#L256-L275 that somehow Jupytext is unable to overwrite.

I believe this is also related somehow to the changes that migrating mybinder https://blog.jupyter.org/updating-defaults-on-mybinder-org-4df41d601955 and the 2i2c hubs https://github.com/2i2c-org/infrastructure/pull/2210/files to use ServerApp instead of NotebookApp. Probably because these changes are the ones that impose the use nbclassic for the classical notebook interface. Unfotunatelly I wasn't able to 'fix' the issue by just reverting the changes there. Maybe I missed something. But going back to NotebookApp should not be the recommended way forward anyway since everything moves the the ServerAPpps' direction. What to do next

Given that this incompatibility is already captured in the relevant upstream projects:

mwouts/jupytext#1051 https://github.com/mwouts/jupytext/issues/1051

jupyter/nbclassic#157 https://github.com/jupyter/nbclassic/issues/157 ,and we also see it for the 2i2c default user image. We should try to provide as much details as possible about what we saw it's happening and/or invest some cycles into upstream work so that those issues to come to a solution.

LIS hub

I believe JupyterLab can be used instead of the classical notebook, with Jupytext, now that we concluded that they work ok toghether on the current hub setup, until the upstream issues get resolved.

— Reply to this email directly, view it on GitHub https://github.com/2i2c-org/infrastructure/issues/2302#issuecomment-1473825235, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAQQHCXUKCP4VQAHLCKMDTW4RPOTANCNFSM6AAAAAAVOWEFWA . You are receiving this because you were mentioned.Message ID: @.***>

GeorgianaElena commented 1 year ago

I've updated the top comment with latest findings in https://github.com/mwouts/jupytext/issues/1051.

Since @matthew-brett confirmed that the workaround described there worked, I will close this issue and track updating the nbclassic version once the fix is released (or using the workaround for now) in the 2i2c hub user image.

Thanks!

matthew-brett commented 1 year ago

Great - and thanks for all your work on this.

2i2c-org / infrastructure