Publish `plum-dispatch-purepython` without C extensions to PyPI

[seeM]

If my benchmarks are accurate (using benchmark.py), cythonising plum.function is at best 2x faster. For our use-case — a high-level framework on top of lower-level libraries like numpy and torch — this isn't worth the loss of stack trace and inspect support.

@wesselb would you consider publishing a build of plum without C extensions to PyPI?

One way to go about this is for setup.py to only add to ext_modules if an environment variable is set (e.g. USE_CYTHON), and for the GitHub Action to do an additional build and publish with that variable set. If you're on board, I'm happy to help where I can!

[wesselb]

Hey @seeM!

Sorry for not replying sooner. I've been in the last stretch of writing up my dissertation, which is now finally done, so I can get back to it.

If Cythonising plum.function only gives a 2x speed-up, perhaps the right choice is to get of Cythonising entirely in the main library. As you say, perhaps the loss of stack trace and inspection support isn't worth it. @PhilipVinc, what are your thoughts on this?

I am taking a break until the end of next week, during which I won't have access to my laptop. After that, I'll be back to work. Once I'm back, I'll setup a few more careful benchmarks to really figure out the benefits of Cythonising. If the benefits are only marginal and if others agree, perhaps we can then decide to just get rid of the compilation step wholly.

[jph00]

Good job with the dissertation @wesselb ! FWIW, I'd very much support making plum cython-free by default.

[seeM]

Congrats @wesselb, great achievement! 😄 I hope you have a super restful break. Please let me know how I can help when you're back

[jmosbacher]

Just came to the issues section to ask this very question. This would be greatly appreciated! I absolutely love this package and want to use it for everything but I also really want my packages to be usable in pyscript so I feel like my parents are going through a rough divorce and forcing me to choose... PS @wesselb congrats on finishing your thesis! this package is truly awesome.

[wesselb]

@jph00, @seeM, @jmosbacher, thank you all! :) I just got back from a lovely break.

It seems that dropping Cython compilation is desirable and comes with a lot of conveniences. Hence, if there are no objections from those involved (@PhilipVinc), I'll simply drop the compilation and release new version shortly.

[wesselb]

@seeM, version 7.1 of the package drops Cython complication and includes a pure-Python wheel. Would you be able to see if the tests pass on your end for this version? (My brain is still operating in holidays mode—just checking I'm not doing something silly. 😅)

[PhilipVinc]

@wesselb I was on holiday and shut down my communication channels.

I'm all for better stack-trace errors. In any case, I use plum's dispatch from within jax.jit frames so I don't really mind about speed.

[wesselb]

@PhilipVinc I hope you had a nice holiday. :) I'm glad you're also on board with this choice! It seems like the superior option, then.

[seeM]

@wesselb glad you had a lovely break. It is indeed working, thank you! 😄

[wesselb]

@seeM that's great to hear! Let us know if there'd be anything we could assist with.