clj-python / libpython-clj

Python bindings for Clojure
Eclipse Public License 2.0
1.06k stars 69 forks source link

V2 #147

Closed cnuernber closed 3 years ago

cnuernber commented 3 years ago
jjtolton commented 3 years ago

Whoaaa huge overhaul! Love it. Especially love the data oriented touches on libpython-clj2.python.ffi.

cnuernber commented 3 years ago

Huge overhaul. I am putting lots of work into making sure that there is far less ephemeral python garbage - so for instance before when you called a function (pyfn 1 2 3 4) the python integers would be linked to the GC in order to get cleaned up eventually - now they are cleaned up just after the function call with no GC intervention. This pathway is much quieter in terms of addref/release logging and much faster than before. This should also lead to considerably more stable behavior as you won't get large batches of objects getting cleaned up at random times.

And getting to the nitty gritty, to call any python function you have to make a tuple of arguments and the python C pathway to set a tuple item steals the reference so if, for instance, you have an integer and you call PyLong_FromLongLong and set that in the tuple you yourself don't have to decrement the reference to the long python object; decrefing the tuple will decref the long value.

Little details like above lead to much smoother and predictable program behavior and I am really looking forward to all of this getting stabilized and for people to start working with the system. They are also why building something like libpython-clj is very hard; reference counting is quite easy to screw up :-).