spiraldb / ziggy-pydust

A toolkit for building Python extensions in Zig.
https://pydust.fulcrum.so/
Apache License 2.0
371 stars 16 forks source link

Alternatives to poetry #144

Open shakfu opened 11 months ago

shakfu commented 11 months ago

Hi,

Really cool project, I'm very pleased to discover it given that I am trying to learn zig.

I'd be interested to explore alternative ways to use ziggy-pydust.

I'm thinking that a single python script (akin to Django's manage.py but self-contained) could drive project creation, project mgmt tasks.py and virtualenv creation. This could be an alternative to using poetry.

I would also like to be able to use my builtin installation of zig v12 rather than having to use v11 if possible.

gatesn commented 11 months ago

Regarding the v11 zig, you should be able to set zig_exe = <path> in your pyproject.toml - although that sort of requires a Poetry pyproject.toml!!

Are you saying you'd like to use the project without any Python-specific build system at all? I do think that's doable, but it would require us building the tools to generate valid wheels and source dists, etc.

shakfu commented 11 months ago

@gatesn Thanks for your reply

Regarding the v11 zig, you should be able to set zig_exe = in your pyproject.toml - although that sort of requires a Poetry pyproject.toml!!

Understood 👍

Are you saying you'd like to use the project without any Python-specific build system at all? I do think that's doable, but it would require us building the tools to generate valid wheels and source dists, etc.

Not at all. I understand the convenience that poetry provides given its wholistic approach, but I am just curious about the hypothetical minimal requirement to run ziggy-pydust in a project. Could it be done, for example, with just a pip install -r requirements.py.. What are the extras beyond that? a virtualenv, zig executable in path, generation of build.zig, ... ? if these are known then I'd like to have a go at creating a script to enable project creation outside of poetry...

gatesn commented 11 months ago

Ah gotcha. So with ziggy-pydust in your environment, the main build script is here: https://github.com/fulcrum-so/ziggy-pydust/blob/develop/pydust/build.py#L20

We may need a small refactor to builzig so it accepts config as an argument instead of loading it itself. (https://github.com/fulcrum-so/ziggy-pydust/blob/7291db36ec6ac2362671acdfd75ec83569f45fbf/pydust/buildzig.py#L36)

That way, you could build your own config object (https://github.com/fulcrum-so/ziggy-pydust/blob/7291db36ec6ac2362671acdfd75ec83569f45fbf/pydust/config.py#L45) and then invoke the build.

Similarly, the pytest_plugin.py might need its own way to load some config.

You should be able to try this out by cloning the repo and then pip install -e ../ziggy-pydust or wherever you've cloned it to.

shakfu commented 11 months ago

Thanks very much for your help and guidance. I'll revert back if I have something worth sharing...

slonik-az commented 11 months ago

Just some thoughts: Python has at least three popular package managers. Besides Poetry they are pip and anaconda (dominant in data science). Providing clear dependencies list would help to adopt ziggy-pydust to other package managers and extend its user base.

alexporter8013 commented 11 months ago

Just some thoughts: Python has at least three popular package managers. Besides Poetry they are pip and anaconda (dominant in data science). Providing clear dependencies list would help to adopt ziggy-pydust to other package managers and extend its user base.

Hello all, I'm new to this repo, but what I would recommend would be to take a look at https://github.com/pypa for "official" build systems in addition to poetry and conda to cover the 1st and popular 3rd party build tools.

shakfu commented 11 months ago

FYI, I have just added a pull request PR-158 which enables building an extension without any build system except zig's:

python -m pydust build --generate-stubs --ext-name 'fib._lib' --ext-path 'fib/fib.zig'

This builds the fibonnaci example in-place with optional stub generation enabled. Note that each --ext-name and --ext-path pair can be repeated for multiple extensions.

shakfu commented 11 months ago

An update: PR-158 has been merged, and as of release .0.12.1 the command line api for the new build subcmd is:

usage: pydust build [-h] [-z ZIG_EXE] [-b BUILD_ZIG] [-m] [-a]
                         [-e EXTENSIONS [EXTENSIONS ...]]

options:
  -h, --help            show this help message and exit
  -z ZIG_EXE, --zig-exe ZIG_EXE
                        zig executable path (default: None)
  -b BUILD_ZIG, --build-zig BUILD_ZIG
                        build.zig file (default: build.zig)
  -m, --self-managed    self-managed mode (default: False)
  -a, --limited-api     use limited python c-api (default: True)
  -e EXTENSIONS [EXTENSIONS ...], --extensions EXTENSIONS [EXTENSIONS ...]
                        space separated list of extension '<name>=<path>'
                        entries (default: None)

The earlier api has changed to the more intuitive form for multiple extensions:

python -m pydust build --extensions 'fib._lib=fib/fib.zig' 'fib._hello=fib/hello.zig'