"Exclude-patterns" option doesn't work as intendet for paired .md files

[MomoInSpace]

TLTR:

If there are two files with the same name but different endings, say file.md and file.ipynb, how can one prioritize to render the output of the .ipynb instead of the empty output of the .md files?

Describe the bug

I'm working in a sphinx project where we want to use myst-nb with paired notebooks. That means that we have folders with files with the same name but different endings:

some_folder
  neat_code_examples.md  
  neat_code_examples.ipynb

We now want to exclude the Markdown files for building the sphinx site by using the following option in the conf.py file:

exclude_patterns += [
    fn.replace(".ipynb", ".md") for fn in glob.glob("**/*.ipynb", recursive=True)
]

This works with nbsphinx, but apparently myst-nb still uses the .md files instead of the .ipynb files, as sphinx doesn't generate output this way. When you delete all paired markdown files we have output. Is there another way to tell myst-nb to prefer one filetype over the other when you have the same filename, just different types? Otherwise it should use the exclude patterns and ignore the files in said array.

Reproduce the bug

Create a sphinx project, eg with

mkdir some_project
cd some_project
sphinx-quickstart
cd source
mkdir some_folder

Modify the index.rst (see below). Then create a paired notebook with the name neat_example in the folder some_folder, it should have a header and some output. Finally, modify the config.py as seen below. If you now switch between nbsphinx and myst_nb in the extensions, you will see, that after building the project, you have an output with nbsphinx, whereas it is missing when using the myst_nb extension.

config.py

# Configuration file for the Sphinx documentation of EasyGems
import glob
#
# This file only contains a selection of the most common options. For a full
# list see the documentation:
# https://www.sphinx-doc.org/en/master/usage/configuration.html

# -- Path setup --------------------------------------------------------------

# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
# import os
# import sys
# sys.path.insert(0, os.path.abspath('.'))

# -- Project information -----------------------------------------------------

project = 'paired_issue'
copyright = '2022, momo'
author = 'momo'

# -- General configuration ---------------------------------------------------

# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
    # "myst_nb",  # MySt for jupyter-notebook files
    "nbsphinx",
]

# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']

# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This pattern also affects html_static_path and html_extra_path.
exclude_patterns = ["**.ipynb_checkpoints"]

exclude_patterns += [
    fn.replace(".ipynb", ".md") for fn in glob.glob("**/*.ipynb", recursive=True)
]

nb_execution_mode = "auto"  # executes only if python cells are not evaluated

nb_execution_excludepatterns = exclude_patterns

# -- Options for HTML output -------------------------------------------------

# The theme to use for HTML and HTML Help pages.  See the documentation for
# a list of builtin themes.
#
html_theme = 'alabaster'

# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']

index.rst:

.. paired_issue documentation master file, created by
   sphinx-quickstart on Wed Jul  6 12:35:04 2022.
   You can adapt this file completely to your liking, but it should at least
   contain the root `toctree` directive.

Welcome to paired_issue's documentation!
========================================

.. toctree::
   :maxdepth: 2
   :caption: Contents:

   some_folder/neat_example   

Indices and tables
==================

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`

List your environment

myst-nb 0.13.2 jupytext_version 1.13.8 python 3.8.3 sphinx 4.5.0

[welcome[bot]]

Thanks for opening your first issue here! Engagement like this is essential for open source projects! :hugs:
If you haven't done so already, check out EBP's Code of Conduct. Also, please try to follow the issue template as it helps other community members to contribute more effectively.
If your issue is a feature request, others may react to it, to raise its prominence (see Feature Voting).
Welcome to the EBP community! :tada:

[chrisjsewell]

Heya, I believe this is a limitation of sphinx, which only stores filenames without their extensions, which means that it cannot discriminate

This works with nbsphinx

I imagine this only because nbsphinx does not incorporate a Mardown file reader. If you used e.g. nbsphinx + myst_parser you would have the same problem.

If you don't want any .md files to be parsed, I think you may be able to set: https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-source_suffix, to not include it, e.g.

source_suffix = {
    '.rst': 'restructuredtext',
    '.ipynb': 'myst-nb',
}

or you can store the paired files in different folders, with https://jupytext.readthedocs.io/en/latest/config.html

[formats]
"notebooks/" = "ipynb"
"markdown/" = "myst"