An efficient method of calling C++ functions from numba/Python using clang++/ctypes/rbc

Quansight-Labs / quansight-labs-site

💻 Development site and blog for Quansight Labs

https://labs.quansight.org

24 stars 44 forks source link

An efficient method of calling C++ functions from numba/Python using clang++/ctypes/rbc #266

Closed pearu closed 2 years ago

pearu commented 2 years ago

Text styling

[ ] The blog is written with plain language (where relevant).
[ ] If there are headers, they use the proper header tags in order to do so (with only one level-one header).
[ ] All links describe where they link to (for example, check the Quansight labs website).
[ ] Any kind of styling that the author uses (for example, bold for emphasis) is consistent throughout the blog.

Non-text contents

[ ] All content is represented as text (for example, images need alt text and videos need captions or descriptive transcripts).
[ ] If there are emojis, there are not more than three in a row.
[ ] Don't use flashing gifs or videos.
[ ] If it were to be read as plain text, the blog still makes sense and no information is missing.

cloudflare-pages[bot] commented 2 years ago

Deploying with Cloudflare Pages

Latest commit:	`25a6502`
Status:	✅ Deploy successful!
Preview URL:	https://6143047c.quansight-labs-site-6s3.pages.dev

View logs

mattip commented 2 years ago

Have you seen the cppyy project? I am not sure how it would work with Numba (there is a mention that it works but not more than that). It is meant to be faster and cleaner than pybind11 but has a runtime dependency, like Numba, on the llvm runtime.

pearu commented 2 years ago

Have you seen the cppyy project? I am not sure how it would work with Numba (there is a mention that it works but not more than that). It is meant to be faster and cleaner than pybind11 but has a runtime dependency, like Numba, on the llvm runtime.

Thanks, @mattip for the reference. I think cppyy is worth exploring with respect to numba/C++ connectivity.

Btw, I have added a section "A list of Python/C/C++ connectivity tools". Let me know if some important project is missed.

mattip commented 2 years ago

Btw, I have added a section "A list of Python/C/C++ connectivity tools". Let me know if some important project is missed.

Thanks, it looks even too complete. Maybe only include the ones that are C only like the Python C APi, cython, cffi?

pearu commented 2 years ago

Btw, I have added a section "A list of Python/C/C++ connectivity tools". Let me know if some important project is missed.

Thanks, it looks even too complete. Maybe only include the ones that are C only like the Python C APi, cython, cffi?

Hmm, the blog post is about C++ connectivity, so C++ projects ought to be relevant (also C projects are relevant as these allow building bridges to C++).

pearu commented 2 years ago

Btw, I cannot see why Cloudflare Pages build fails. Could anyone post the build failure messages?

rgommers commented 2 years ago

Btw, I cannot see why Cloudflare Pages build fails. Could anyone post the build failure messages?

I don't seem to have access. That's not great - I don't even know if that's supposed to work (we recently switched to CloudFlare from Netlify, but not very smooth sailing yet).

I usually build locally, should be a matter of creating a new conda env with Python 3.8, pip install from the requirements file in the repo root, and running nikola auto.

rgommers commented 2 years ago

Given that cxx2py.py seems useful and needs a stable link long-term, should it be moved from https://github.com/pearu/pearu.github.io/blob/main/cxx2py/cxx2py.py to a separate repo under Quansight-Labs, or should it be part of the RBC repo maybe?

EDIT: ah wait, there's also cxx2py_libfoo.cpp. If you prefer to keep them where they are that may be fine, as long as the links are stable.

rgommers commented 2 years ago

Before publication this should incorporate  so the blog front page shows the first 1-2 paragraphs only. Besides that, it looks nice a great post.

pearu commented 2 years ago

@rgommers I have added the TEASER_END comment as well as copied the referenced files under quansight-labs-site for maximum stability. In case we'll continue working with the cxx2py.py idea, I think it should be turned into a separate Python package anyway.

tonyfast commented 2 years ago

i built this locally and had some nits. i removed the first two headings for the sake of the blog roll in the index. the title still shows up because it is in the front matter.

rgommers commented 2 years ago

tonyfast merged commit 3f51cc7 into main 3 hours ago

I still had a bunch of review comments on the way, let me submit them now - @pearu feel free to ignore the ones you don't think are relevant, now that the PR is merged.

pearu commented 2 years ago

@rgommers thanks for your feedback! I have addressed your comments in https://github.com/Quansight-Labs/quansight-labs-site/pull/270 .