Is the ServiceX Client a library or an application?

:wave: At the moment on the 3.0_develop branch the servicex package is being packaged as a Python application and not a Python library. If this is intended, this is not being made clear from the project README, and it would help a lot to be made as clear as possible to the user that they should not expect that servicex should ever be installed in an existing Python virtual environment with other tools but that it should exist in its own virtual environment by itself. I think that the idea was to have the client work with the rest of the IRIS-HEP tooling in the same virtual environment, which in that case it should be packaged as a library, allowing for it and its dependencies to be resolved into a deployment lock file created by the user (or the site admin that is maintaining an environment for users).

At the moment this makes servicex v3.0 quite difficult to work with and to do developmental testing in upcoming CPython releases as the poetry restrictions are enforced at the packaging level and so the end user has no ability to alter these requirements.

@henryiii has a good layout of all of these issue here in his Should You Use Upper Bound Version Constraints? blog post.

Main question: From the servicex dev team's perspective how should we as users be thinking about servicex: as an application or library? If the goal is a library, can the dev team revisit their approach of using poetry's incredibly restrictive caret requirements notation which ends up causing lots of pain outside of lock file usage (as covered in Henry's blog).

Aside: Requiring installs of metapackages like jupyter

https://github.com/ssl-hep/ServiceX_frontend/blob/b53c8e4497cf7dde8c8dc93da78eb10fe0ef51cd/pyproject.toml#L36

and pinning them to a particular version also is confusing as it brings a large number of dependencies into the runtime environment that aren't explicitly needed. Pinning to jupyter is also the wrong approach, though it probably isn't obvious at first (wasn't to me for a few years). The jupyter metapackage isn't updated (note last release was in 2015) and instead pinning of notebook, jupyterlab, and any other packages it brings along is the suggested approach to provide API stability.

Second aside: If the main selling point of poetry (not sure as there are no issues or PRs discussing the adoption) was lock files, then I would point to uv doing lock files in a similar system to pip-tools, but far faster.

Related Issues and PRs:

ssl-hep / ServiceX_frontend

Is the ServiceX Client a library or an application? #346