Reorganize repo to `src-layout`, migrate to `hatch` build system and `pyproject.toml`

[mahendrapaipuri]

This PR attempts to reorganize repo to src-layout, migrate the package to hatch build system which is being used in the Jupyter ecosystem and also address PEP621 by moving all the package metadata to pyproject.toml.

New organization:

Only important folders are listed in the tree

.
├── binder
├── demo
├── docs
├── jupyterlab
│   ├── jupyter-config
│   ├── jupyterlab_jupytext
│   └── packages
│       └── jupyterlab-jupytext-extension
│           ├── src
│           └── style
├── src
│   ├── jupytext
│   └── jupytext_config
└── tests

Breaking changes:

• The organization of repository has been changed to src-layout to put all core modules of jupytext in src/ folder
• A separate new module jupytext_config has been created with same behaviour as jupytext-config console script
• The JupyterLab extension jupyterlab_jupytext has been moved into jupyterlab/ folder including all jupyter related config files. jupyterlab_jupytext will be distributed as another python module along with jupytext, jupytext_config. All the jupyter_server and jupyterlab related extension registration has been moved into jupyterlab_jupytext module.
• To trigger extension build process, we need to use HATCH_BUILD_HOOKS_ENABLE=true env var instead of current BUILD_JUPYTERLAB_EXTENSION=1 env var for local dev.

TLDR; Instead of distrtributing a single jupytext package (which is the current case now), we split it into three jupytext (core), jupytext_config (config) and jupyterlab_jupytext (Jupyter server and lab extension). Building of labextension is disabled by default and we can enable it using HATCH_BUILD_HOOKS_ENABLE=true env var passed at pip install .

List of changes:

• Minimum Python version bumped to 3.8 inline with JupyterLab 4
• sphinx-gallery dependency has been removed
• All package metadata has been moved to pyproject.toml
• Unused tox config has been removed
• Lab extension's dependencies have been bumped to JupyterLab 4
• Lint config has been added to TS extension and a TS lint test in CI build step has been as well
• CI workflow file has been split into multiple step files
• jupyter-maintainer-tools are used to setup python and node in CI workflows
• Added check-release.yml workflow from jupyter-releaser
• A Changelog generated from github-activity has been added

Misc:

I have added check-release.yml workflow from jupyter-releaser just to check if it works with the current organisation. If you are keen to move to jupyter-releaser, the workflow would be to manually bump the versions in a PR and then use prep-release and publish-release workflows which will publish python and npm packages.

Current organization does not let us to publish npm packages from jupyter-releaser. If we move jupyterlab/package.json and its auxillary config files to the root of the top, we can build and publish npm packages from jupyter-releaser workflows as well.

Todo:

• Documentation changes to be made once we agree on organizational changes to the repo.

Closes #1076 Closes #1114 Closes #1115 Closes #1135

[mahendrapaipuri]

@LecrisUT Cheers for the review!! I will wait for @mwouts as well before making any further changes, if that is ok.

[mwouts]

Hi @mahendrapaipuri , @LecrisUT , I am sorry I have too little time tonight to review this in more details, I'll do more tomorrow.

Is the local build supposed to work with just python -m build? Locally it fails with this message:

OSError: Error getting the version from source `regex`: file does not exist: src/jupytext/version.py

(but the file does exist)

Thanks again for making this happen!

[LecrisUT]

Is the local build supposed to work with just python -m build? Locally it fails with this message:

OSError: Error getting the version from source `regex`: file does not exist: src/jupytext/version.py

Yes, that's the intended interface. Not sure what's causing the error. I can help debug this issue in the weekend if it persists.

[mahendrapaipuri]

Closed in favour of #1140

@mwouts @LecrisUT Lets move the discussion to #1140