search

facebook / CacheLib

Pluggable in-process caching engine to build and scale high performance services
https://www.cachelib.org
Apache License 2.0
1.18k stars 256 forks source link

Py bindings #208

Closed journaux closed 1 year ago

journaux commented 1 year ago

Is your feature request related to a problem? Please describe. I'm frustrated duplicating & piping data over local sockets vs direct i/o

Describe the solution you'd like python bindings for local/remote workloads operating on computer vision queues

Describe alternatives you've considered pipe data over local sockets

Happy to contribute bindings using pybind11 if repository owners agree on plan/design

jaesoo-fb commented 1 year ago

Hi @vitruvvius

Yeah, we don't support python binding yet; only binding supported other than C++ is rust so far and we welcome any contribution to support a new binding for python.

Anyway, this looks like not an issue but a discussion for a new feature. If needed, could you start a new discussion?

journaux commented 1 year ago

Yes - fyi the current rust bindings are broken for external contributors

jaesoo-fb commented 1 year ago

Yes - fyi the current rust bindings are broken for external contributors

I haven't checked it, but could you elaborate more on how it is broken? It would be great if you can create an issue with some repro steps?

journaux commented 1 year ago

@jaesoo-fb https://github.com/facebook/CacheLib/issues/92

Start a docker container. Build cachelib (./contrib/build) Build rust bindings (cd cachelib/rust; cargo build)

Why? (i) Dependency on folly bindings (folly::StringPiece<_>) not present (ii) Dependency on FB internal test crate not present

Find my latest comment @ the bottom of thread.

My preferred design would be autocxx (c++ variant of bindgen) - or any automated system - to generate raw bindings off source instead of hand-writing cxx.

We want to build RDMA WALs w/ cachelib in compute nodes

journaux commented 1 year ago

Related lines to (i) & (ii)

https://github.com/facebook/CacheLib/blob/e679b72d5b78c51f1cd3a250ad4918250af8ddc9/cachelib/rust/src/lib.rs#L37 https://github.com/facebook/CacheLib/blob/e679b72d5b78c51f1cd3a250ad4918250af8ddc9/cachelib/rust/src/lib.rs#L44 https://github.com/facebook/CacheLib/blob/e679b72d5b78c51f1cd3a250ad4918250af8ddc9/cachelib/rust/src/lrucache.rs#L1205 https://github.com/facebook/CacheLib/blob/e679b72d5b78c51f1cd3a250ad4918250af8ddc9/cachelib/rust/src/cachelib.h#L24 https://github.com/facebook/CacheLib/blob/e679b72d5b78c51f1cd3a250ad4918250af8ddc9/cachelib/rust/src/cachelib.cpp#L26