File not found in collaborative mode Jupyterlab v4

[Podolyakofs]

Good day, don't know if this bug is for jyputer-h5web or for jupyter-collaboration

In jupyterlab v4 with collaborative mode got file not found because of adding "RTC:" to filepath - for example "File not found: 'RTC:discharges/2023/202306/20230629/T/1230T 2023-06-29 13:03:54.nx"

H5Web("discharges/2023/202306/20230629/T/1230T 2023-06-29 13:03:54.nx") - works

To Reproduce

Try to open hdf5/nexus file in jupyterlab with jypyter-collaboration enabled

Expected behaviour

Open file without error

Screenshots

For example csv works

but to open from python (with open(..)) "RTC:path" needs to be changed to "path")

Context

• OS: Ubuntu 22.04
• Browser: Chrome
• Version: 114
• JupyterLab version: 4.0.2

Extension lists

Paste the output from running `jupyter labextension list` and `jupyter serverextension list` from the command line here.
You may want to sanitize the paths in the output.

 jupyter_lsp  enabled 
    - Validating...
      jupyter_lsp 2.2.0 OK
    jupyter_nbextensions_configurator  enabled 
    - Validating...
      jupyter_nbextensions_configurator 0.6.3 OK
    jupyter_server_ydoc  enabled 
    - Validating...
      X is jupyter_server_ydoc importable?
    jupyterlab  enabled 
    - Validating...
      jupyterlab 4.0.2 OK
    jupyterlab_git  enabled 
    - Validating...
      jupyterlab_git 0.41.0 OK
    jupyterlab_h5web  enabled 
    - Validating...
      jupyterlab_h5web 10.0.0 OK
    nbdime  enabled 
    - Validating...
      nbdime 3.2.1 OK

        jupyter-vuetify v1.8.10 enabled  X
        jupyterlab-h5web v10.0.0 enabled OK (python, jupyterlab_h5web)
        nbdime-jupyterlab v2.2.0 enabled  X
        jupyterlab_pygments v0.2.2 enabled  X (python, jupyterlab_pygments)
        jupyter-vue v1.9.2 enabled OK
        jupyter-matplotlib v0.11.3 enabled OK
        @jupyter-widgets/jupyterlab-manager v5.0.7 enabled  X (python, jupyterlab_widgets)
        @jupyter/collaboration-extension v1.0.1 enabled OK (python, jupyter_collaboration)
        @bokeh/jupyter_bokeh v3.0.7 enabled  X (python, jupyter_bokeh)
        @jupyterlab/git v0.41.0 enabled  X (python, jupyterlab-git)

Also checked on new vm:
 jupyterlab-h5web v10.0.0 enabled OK (python, jupyterlab_h5web)
        jupyterlab_pygments v0.2.2 enabled  X (python, jupyterlab_pygments)
        @jupyter/collaboration-extension v1.0.1 enabled OK (python, jupyter_collaboration)

[lrlunin]

Can confirm the issue above.

[loichuder]

Thanks for the report !

The issue may be due to the patching I did to prevent the fetching of the file.

Hopefully, this should no longer be needed since we now only support JLab4 that has the file format null. So, it may be as simple as removing the patching and set the file format to null in HDF5_FILE_TYPE. To be tested :shrug:

I will gladly accept a PR or a report on whether this fixes the issue. Otherwise, this will have to wait a bit until I find time to work on it.

[loichuder]

On second thought, that may be completely unrelated :joy:

I am wondering how Y.js (the tech powering the sharing of files in collaboration mode) deals with binary files. I understand it can work with plain text such as CSV but that may be another story with HDF5 files.

[lrlunin]

I am wondering how Y.js (the tech powering the sharing of files in collaboration mode) deals with binary files.

I was testing the new RTC version the whole day yesterday and still have the issue that the created .ipynb files are not saved :D (Was problem on my site)

I would expect that people will rather rarely edit HDF5 files in the collaborative mode in way like people edit .ipynb files. Maybe we should request to handle the "plain text" files (like .txt, .scv etc) with Y.js and then just ignore all other files formats which they cannot handle correctly.

[lrlunin]

Hello, @loichuder

I guess this issue is realated to this issue: https://github.com/jupyterlab/jupyter-collaboration/issues/183#issuecomment-1697679929

I believe they argue that need to be fixed on the extension site.

[loichuder]

Thanks for following that up @lrlunin, it indeed sheds some light on the issue.

I will wait for JLab maintainers' answer to my comment to see if JLab has a consistent way of accessing custom drives (that is used for collaborative mode). If not, I will have to patch something together myself.

[betolink]

I just ran into this issue, is there a known workaround? I'm using JLab 4 and the latest jupyterlab-h5web versions.

[loichuder]

Sorry, it has been a while before I got back to this. Next release should include a quick and dirty fix.

[loichuder]

I just released jupyterlab-h5web 12.0.0 that includes the aforementioned fix.

Since I don't use RTC myself, I was only able to do basic checks and I am not convinced it was the best way of solving the issue.

If anyone can test and report, that would be greatly appreciated. I'll leave this issue open and you are welcome to post further comments if additional problems are encountered.

[Podolyakofs]

Checked the last version - got same error

File not found: 'RTC:discharges/2024/202402/20240226/M/647M 2024-02-26 13:56:07.nxs'

    tornado.web.HTTPError: HTTP 404: Not Found (File not found!)
[W 2024-02-26 19:28:38.604 ServerApp] 404 GET /h5web/meta/?file=RTC:discharges%2F2024%2F202402%2F20240226%2FM%2F647M+2024-02-26+13:56:07.nxs&path=%2F (8d227a79e2e046ffa52fec4815b54a01@192.168.195.209) 2.36ms referer=http://hdfcon.tkmk:8888/lab/tree/RTC%3Adischarges/2024/202402/20240226/M

Also in 12.0.0 get In previous version (11.1) there is no error

[loichuder]

Thanks for checking @Podolyakofs

I'll have a second look

[loichuder]

It seems to run fine on my end. I'll try to reproduce the error with the same parameters as you.

@Podolyakofs could you give me:

1. The list and versions of installed packages in the environment (pip freeze/list)
2. The way you start and connect to the RTC instance of JupyterLab
3. The file you are using (or a stripped down version, as long as it shows the error Expected Group)

That would be very helpful for me to debug, thanks.

[Podolyakofs]

Checked on new machine - looks like everything works!

[loichuder]

Great to read :relieved: !

I'll keep following discussions on the JLab repo in case there is a better way of fixing it

[betolink]

I'm using the latest version of H5Web and I still can't open the files if I use the collaborative mode =( is there something extra I should do or it's a matter of waiting for future releases?

[Podolyakofs]

I'm using the latest version of H5Web and I still can't open the files if I use the collaborative mode =( is there something extra I should do or it's a matter of waiting for future releases?

How do you start collaborative mode? As I understand in jupyterlab v4+ the only need is the jupyter-collaboration-extension installed/enabled and --collaborative flag is not used anymore during startup

[loichuder]

Sorry @betolink , I missed your comment :disappointed:

The release is done: it should work with version 12.0.0 of the extension.

As @Podolyakofs, I am also curious to know how you start the collaborative mode.

[Podolyakofs]

As @Podolyakofs, I am also curious to know how you start the collaborative mode.

Before Jupyter v4 (for jupyter v3) I start with

jupyter lab --no-browser --ip "*" \
             --notebook-dir /home/main/jupyter 
             --collaborative

after they separate collaboration into plugin (jupyter v4) I start only with:

jupyter lab --no-browser --ip "*" \
             --notebook-dir /home/main/jupyter 

(and enabled extention)

Here is the issue about it https://github.com/jupyterlab/jupyter-collaboration/issues/194 (that flag now is enabled by default if plugin is installed)

I think there can be some errors if use that flag in J v4 with RTC extention enabled

Screenshots for J v4:

[loichuder]

If there is no more reports, I'll close this as completed. As mentioned, I will still check if another "more official" way of solving this emerges