rpav
commented
6 months ago cpp
So the preprocessor thing will always be a huge hack, because the preprocessor is a huge hack: you've got arbitrary textual substitutions you need to somehow make sense of. There might be better heuristics about what to pick, but there haven't been many issues of "this doesn't capture a preprocessor constant we need." It works, and I'm not sure much better is possible.
(For instance, consider there can be #define FOO_SIZE (2*sizeof(Foo)), or #define HAS_FOO __weird_intrinsic_magic; simple parsing will never resolve these, but c2ffi will, and they're often the definitions you really want.)
libclang
Libclang never appeared to give sufficient information about the source; it's possible I either misinterpreted the docs or that's changed.
But generally, it's less than an hour to maintain per version. So why does it take me forever to update sometimes? See below.
ObjC etc
ObjectiveC support is limited because I have no reference; I don't know ObjC and I have nothing to test/verify/use it with. C++ support is limited and could be better, but this is mostly some additional info (I think public/private/protected still isn't there). I would certainly welcome patches for ObjC.
rewrite
When I think "rewriting," I don't mean tossing the source and starting over: there doesn't seem to be much reason, all the parts are working and in order, and do what would need done anyway.
I mean going back through all the source and making it cleaner, more portable C++. For instance, nlohmann/json, cxxopt instead of getopt, and generally less C-like and more modern C++ code. This makes working with it nicer when it comes to maintenance, and would probably make building with MSVC possible (as long as the clang libraries are available).
Of course there are always other features that would be nice, but these are mostly independent modular changes:
- Better json code, making it easier to extend; this is the worst at a glance and the least fun to work with. C doesn't change much but this might help C++/ObjC.
- Built-in filtering / treeshaking.
below
But here are the real actual issues:
- New versions of llvm/clang are annoyingly inconsistent to obtain: Which distro has them and when varies wildly, and they are often not what I'm using at any given time.
- Distro-built versions aren't always consistent/working, though this has been less of an issue recently.
- My personal dev setup hasn't been the greatest the past few years (though this will be solved after next week >:), so among other things building clang has been out of the question.
- Few of the users of c2ffi seem to be C++ devs and therefore contribs tend to be low. (And therefore doubly-thanks here to all those who have!)
For instance, right now, I couldn't work on the 18.x branch on Linux, as there doesn't appear to be any distro with packages for it. (Changes were minimal, so 17->18 porting would be easy enough, but it's another place the problem appears.)
What really needs done:
- Alpine + docker/podman, and/or AppImage: Alpine linux seems to have the most consistently up-to-date clang/llvm
- Testing/CI/distribution based around this
This basically solves the "wait until Arch has it" problem. It does require a bit of infrastructure building, but this is mostly editor/container configuration, but should minimize the biggest issue today, i.e. "when is c2ffi going to get updated."
@rpav mentioned in a post, and in the README, about the preprocessor handling being a huge hack, and a rewrite of the system may be in order. The timing may be a happy coincidence, as I was participating in another CFFI discussion with the CCL folks, who are embarking on their own version of a CFFI generator.
In that discussion, I suggested using c2ffi for theirs, based on resource constraints. CCL is only barely surviving on life support, and it didn't seem to me a wise use of scarce resources to reinvent the wheel, but you're comments have got me reconsidering the fundamental basis of that assumption.
In that discussion, the main reason for developing a new ffi generator were:
As well as a lack of ObjectiveC support. The basic architecture of c2ffi seems fine to me, but there is a lot hassle with clang/llvm to make it work.
Given your experience over the last few years developing and maintaining c2ffi, I wonder if you'd be able to comment? If you were going to rewrite c2ffi, what would you do differently this time around?
P.S. If you have the ability to move this into a discussion thread rather than an issue, it probably is better discussed there.