Is this project abandoned?

[kat-co]

There is what looks like it might be a fork (it's 36 commits ahead): https://github.com/snmsts/burgled-batteries3. At the time of this writing, there was a commit 26 days ago. The last commit here was 3 years ago.

Common Lisp needs a solid CFFI wrapper around CPython so that it can interoperate with the large and popular Python ecosystem. If you need to abandon this project for whatever reason, please be a good steward and solicit for a new maintainer, or endorse a fork so that there is no confusion where people should direct their efforts to.

Thank you for creating this project. <3

[pinterface]

Thanks for bringing this question up. b-b occupies enough space in my mind to periodically surface in an "Oh man, I need to work on that!" sort of way, complete with extensive guilt for not doing so, but obviously it's been a while since I've really done anything with it.

That's for a few reasons:

1. Life keeps me busy.
2. I'm not actively working on anything which needs this.
3. I'm no longer convinced smashing multiple interpreters into a single runtime is the best idea.

Point 3 deserves a bit of expansion: The basic problem with smashing things together like b-b does is that it's really hard to reconcile all the differences. CPython and the Lisp environment can disagree on pretty low-level things like signal processing, timeouts, and floating point traps (e.g., see #13). On top of that, the languages have some high-level disagreements (for example, Python and CL disagree on how to resolve a class' inheritance tree, which may be resolvable on the Lisp side, but is bound to trip up the one person who manages to run into the difference). At the moment, at least, I'm much more sold on the idea of using job queues to shuffle pieces around to the languages and libraries most appropriate to solve a problem. As always, though, there's an engineering trade-off betwixt the two extremes.

Which is a long-winded way of saying that, yeah, I have not been a good maintainer these last few years, that's sadly unlikely to change in the foreseeable future, and you're right that I should hand off the project to somebody who can do a better job of moving it forward. If someone would like to take it over, or even if they just want me to point to their fork, I am happy to do so; and @snmsts does certainly seem like the best candidate for that.

[kat-co]

What a timely, and great response, thanks @pinterface! I think every developer empathizes with the issue of maintaining something when you're not using it, and when life gets busy. No sweat there!

Are the issues between the competing runtimes intractable, or only tediuous? Could they be encapsulated and surfaced in clear ways? I am reminded of this talk I attended last year which discusses using so-called "type preserving compilation". The talk really focused on types, and type-checking, but I wonder if runtime differences could be handled with annotations as well. As we have macros in CL, we could build a mini-DSL to decorate calls and types with to instruct the runtimes in how to cooperate (which is expressly what the speaker does not want to do, but we are trying to solve a smaller problem here).

Or, a larger question, should the community be putting their effort behind something like WASM for this kind of interoperability?

Regarding your job-queue point, I have also been looking at https://github.com/bendudson/py4cl which uses streams to communicate between separate processes. I'm not sure if this is exactly what you have in mind, but I'm wondering if you endorse this approach.

[pinterface]

While I'm doubtful issues with competing runtimes could be solved completely transparently, which is always my ideal, they can certainly be surfaced where relevant (see #13, again).

It might be an interesting project to investigate whether one could feasibly swap fp flags and signal handlers and whatever else whenever switching between the Lisp and Python sides. I assume the performance hit would make that so slow as to be not worthwhile on every FFI call, but it may be good enough to relegate it to a higher level API that would wrap sections of FFI code. I'll leave that as an exercise to the interested reader. :)

I have no opinion on WASM relevant to this project. Whatever one might do with WASM would be solving a completely different problem. General language interop is a useful problem to solve, no doubt (see: the rather dead Parrot VM, or the many languages that run on the JVM), but well outside the scope of this project.

py4cl is certainly interesting! I think there's room for a multitude of approaches, and which approach is best-suited depends immensely on the particular project being worked on. The embedded FFI approach used by burgled-batteries gives you access to everything CPython has to offer, from Lisp, particularly low-level things where you might otherwise be stuck writing C. cl-python is a Python compiler, giving you a much smaller section of the Python world (no C modules being a biggie), but a much cleaner integration. py4cl gives you a well-grounded approach, keeping the runtimes separate and shunting data back and forth. All those approaches have merits, and drawbacks.