Switch from setup.py to pyproject.toml

[afuetterer]

The pyproject.toml file is decribed in PEP 621. It is the recommended way to package Python projects these days.

Please have a look at my fork's version of this file: https://github.com/afuetterer/BERTopic/blob/1969-toml/pyproject.toml

You are still able to do a local pip install . or pip install bertopic[gensim] from pypi.

Would you accept a PR to convert BERTopic to use pyproject.toml? Please let me know what you think.

Refs:

• https://packaging.python.org/en/latest/tutorials/packaging-projects/
• https://packaging.python.org/en/latest/guides/modernize-setup-py-project/
• https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html

[MaartenGr]

Thanks for the suggestion! What would you think is the main added benefit of doing so in the context of this package? Also, I remember it did not support editable installs a while back. Any idea if that's still the case?

[afuetterer]

First of all: Editable installs are possible, I use them all the time.

I think the added benefits would be:

• easier definition of entrypoints, e.g. CLI
• ability to add additional tooling config in the same file, so the complete project configuration can live in one place
• easier definition of (nested) dependency groups

e.g.

dev = [
    "bertopic[docs,test]", # <- no repeating of mkdocs, pytest etc
]
docs = [
    "mkdocs==1.5.3",
    "mkdocs-material==9.5.18",
    "mkdocstrings-python==1.10.0",
    "mkdocstrings==0.24.3",
]
test = [
    "pytest>=5.4.3",
    "pytest-cov>=2.6.1",
]

• ability to use Dependabot for getting scheduled dependency updates (also possible with requirements.txt, but not setup.py)

From llama3's point of view:

• Consistent configuration: pyproject.toml is a TOML file that can be used to configure your package, including its name, version, and dependencies. This provides a consistent way to manage package metadata.
• Improved readability: The TOML format is more readable than the Python code in setup.py. This makes it easier for developers to understand the configuration of their packages.
• Better support for complex configurations: pyproject.toml supports more complex configurations, such as conditional dependencies and custom build steps. This allows you to manage more complex package dependencies and builds.
• Integration with other tools: pyproject.toml is used by other tools, such as Poetry and pipenv, to manage Python packages. This provides a consistent way to manage packages across different development environments.
• Improved security: The use of TOML files for configuration reduces the risk of code injection attacks that can occur when using Python code in setup.py.

What do you think? Should I submit a PR?

[MaartenGr]

First of all: Editable installs are possible, I use them all the time.

Ah, you're right! It seems that the last time I tried was quite a while ago...

What do you think? Should I submit a PR?

Yeah, sounds great! Would be a nice experience working with it in this project. It's not related to pyproject.toml specifically but at some point, I would love to have some sort of pip install bertopic[minimal] that only contains the very minimal amount of dependencies (even removing HDBSCAN and UMAP). But that is a bit out-of-scope and would require major changes...

From llama3's point of view:

I generally tend to steer away from LLM-based opinions unless they are backed by expert-opinions. But thanks for sharing it!

[afuetterer]

Alright, let me submit a PR then. I just "translate", what is given in the setup.py, modifications of dependency groups should be a separate endeavor, I think.

I found some of the points from llama3 valid, and just wanted to add them here for "completeness". For example I think the "readability" point is fair.

[afuetterer]

Done via #1978