holoviz-dev / nbsite

Build a tested, sphinx-based website from notebooks
https://nbsite.pyviz.org
BSD 3-Clause "New" or "Revised" License
28 stars 14 forks source link

The FixNotebookLinks fails to handle links which are not in the same folder as the notebook with the links #297

Open fohrloop opened 5 months ago

fohrloop commented 5 months ago

I found this out while working on https://github.com/holoviz/holoviews/issues/6086. See the details and MWE / reproducing instructions from there. In short, the holoviews documentation build process has fairly large amount of warnings like this (See: https://github.com/holoviz/holoviews/issues/5676).

 WARNING: 'myst' reference target not found: SomeFile.ipynb
 WARNING: 'myst' reference target not found: OtherFile.rst

In my tests all of the warnings I checked corresponded to a broken link in the documentation. I checked output of one holoviews docs build run, and the number of broken links could be 185:

niko@niko-ubuntu-home:~/tmp$ cat holoviews-logs.txt | grep "WARNING: 'myst'" | wc -l
185

Some (38) of them have also an anchor (#) in them, like

/home/niko/code/holoviews/doc/getting_started/5-Live_Data.ipynb:240004: WARNING: 'myst' reference target not found: ../reference/index.html#streams
/home/niko/code/holoviews/doc/user_guide/03-Applying_Customizations.ipynb:230060: WARNING: 'myst' reference target not found: #Coding-style-guide

and the root cause for these might be different

Demo

Using the "bottom-up" MWE I could reproduce the behavior locally with the latest nbsite version (from git). Using following file structure:

📁 examples/
├─📁 user_guide/
│ ├─📄 03-Applying_Customizations.ipynb
│ └─📄 04-Style_Mapping.ipynb
└─📁 getting_started/
  └─📄 2-Customization.ipynb

And writing in one cell of 03-Applying_Customizations.ipynb the following:

- Here is a link to [style mapping](04-Style_Mapping.ipynb)
- Here is a link to [customization](../getting_started/2-Customization.ipynb)

and adding

These prints ```python # nbsite/nbbuild.py:FixNotebookLinks def preprocess_cell(self, cell, resources, index): if cell["cell_type"] != "markdown": return cell, resources matches = re.findall("\[.+\]\((.+\.ipynb)\)", cell["source"]) print("\n\nSTARTING\n--------\n") print("Notebook", self.nb_path, matches) print("old source: ", cell["source"], "\n") for match in matches: for ft in self.file_types: file_path = os.path.join(self.nb_path, match[:-5] + ft) if os.path.isfile(file_path): cell["source"] = cell["source"].replace( "(%s)" % match, "(%s)" % (match[:-5] + ft) ) break # Try unnumbered path num_name = os.path.basename(file_path) name = re.split(r"^\d+( |-|_)", num_name)[-1] unnumbered_path = file_path.replace(num_name, name) if os.path.isfile(unnumbered_path): cell["source"] = cell["source"].replace( "(%s)" % match, "(%s)" % name ) break print("new source: ", cell["source"], "\n") return cell, resources ```

I could see this in the stdout:

STARTING
--------

Notebook /home/niko/tmp/minimal-doc-error/doc/user_guide ['04-Style_Mapping.ipynb', '../getting_started/2-Customization.ipynb']
old source:  - Here is a link to [style mapping](04-Style_Mapping.ipynb)
- Here is a link to [customization](../getting_started/2-Customization.ipynb) 

new source:  - Here is a link to [style mapping](Style_Mapping.rst)
- Here is a link to [customization](Customization.rst) 

and this in the nbsite build (or sphinx-build) output:

/home/niko/tmp/minimal-doc-error/doc/user_guide/03-Applying_Customizations.ipynb:20003: WARNING: 'myst' reference target not found: Customization.rst

and this in the browser (missing link):

image

I am working on a fix and will supply a PR. This should fix quite many (potentially hundreds if ~150 in just holoviews) broken links from all of the holoviz projects documentation using the notebook directive of the nbsite package.