Set up standard Python packaging

[akx]

This PR sets up modern standard Python packaging using Hatch's Hatchling build backend.

Major changes are:

• the introduction of a standard pyproject.toml file
• cotainr now has a standard console script entrypoint, so bin/cotainr is no longer technically needed
• extra requirements (test, docs) are now specified in the pyproject.toml file (so installable via pip install -e .[test,docs])

This makes Cotainr installable and available directly via e.g.

pip install git+https://github.com/DeiC-HPC/cotainr.git

(or a built wheel – there's a GHA workflow for that too):

~ $ docker run -it python:3.12 bash
root@27974cb842d6:/# pip install git+https://github.com/akx/cotainr.git@packaging
Collecting git+https://github.com/akx/cotainr.git@packaging
Successfully built cotainr
Installing collected packages: cotainr
Successfully installed cotainr-2023.11.0
root@27974cb842d6:/# cotainr
usage: cotainr [-h] {build,info} ...

Build Apptainer/Singularity containers for HPC systems in user space.

options:
  -h, --help    show this help message and exit

subcommands:
  {build,info}
    build       Build a container.
    info        Obtain info about the state of all required dependencies for building a container.
root@27974cb842d6:/#

Please see https://github.com/akx/cotainr/pull/1 for CI results in my fork.

[eskech]

Just a short comment. @Chroxvi is on vacation and is back in two weeks. I really want him to do the review. Sorry for letting you wait until he is back.

[akx]

Gentle nudge (as I too am back from vacation) :)

[Chroxvi]

@akx Thanks a lot for this contribution - and sorry for not getting back to you any sooner.

I definitely agree that we need a proper modern packaging setup for cotainr. Actually, I was working on something very similar in https://github.com/DeiC-HPC/cotainr/compare/main...pip_package, but got distracted by other tasks before I managed to finish it up. My goal was to not only have a proper packaging setup, but also to have an automatic pipeline for building wheels and publishing them on PyPi. I am happy, though, to start with just merging a proper packaging setup, and then revisiting the PyPi publishing in another PR.

Since you work in this PR is almost identical to what I was trying out in https://github.com/DeiC-HPC/cotainr/compare/main...pip_package, we need to consolidate those two branches into a single approach to this. I don't have much experience with all the different available Python packaging tools, nor any strong opinion about specific tools to use. I chose to base my work on "flit" since it seems to be the simplest way to build pure Python packages. Would you recommend "hatchling" and "build" over "flit"? If so, why? Also do you have any other valuable insights that we should consider when implementing this Python packing setup?

[akx]

[...] but got distracted by other tasks before I managed to finish it up.

Story of my life! 😂

My goal was to not only have a proper packaging setup, but also to have an automatic pipeline for building wheels and publishing them on PyPi. I am happy, though, to start with just merging a proper packaging setup, and then revisiting the PyPi publishing in another PR.

You'll be glad to hear that PyPI publishing is practically trivial to add on top of this; a CI step like this would automagically publish to PyPI when a tag is pushed.

I chose to base my work on "flit" since it seems to be the simplest way to build pure Python packages. Would you recommend "hatchling" and "build" over "flit"? If so, why?

build is the canonical, standard, correct Python build frontend. It's orthogonal (or well, on a different layer) c.f. Flit or Hatch. (You can actually use it with Flit-based packages too, since flit_core is a PEP517 build backend.)

I prefer Hatch (its build backend being hatchling, like flit contains extras over flit_core) because I'm very familiar with it, it has sane defaults, the developers are friendly and responsive on GitHub, it's being maintained, etc.

Also do you have any other valuable insights that we should consider when implementing this Python packing setup?

Well, for one, I would heavily consider getting rid of the explicit bin/cotainr wrapper (touching sys.path is 99% of the time a bad move), since a well-behaved wrapper will be provided by the installed package anyway. I didn't want to do that here, though, since it could be a breaking change for some users.

[akx]

Rebased post #56 btw.