Improve how libraries are located and various cleanup

[tarsius]

Hi Tom, it was great talking to you at FOSDEM. I have finally gotten around to playing with this ffi and I'm liking it a lot but I immediately ran into issues.

I think it has a lot of potential but also needs some work to fulfill its potential. Even though this has been sitting around mostly unused for five years now, I hope that you are soon going to put in some more work to make it more usable. It seems to me that the hard parts are done but that the polish is lacking.

I would like to help with that effort starting with this pull-request. It comes with assorted cleanup, but the main improvements concern how libraries are located. That is done in the last three commits.

It's probably not a good idea in the long run to re-purpose load-path for C libraries as well but at least it is a step up from forcing users to place C libraries inside this repository or to bind default-directory around code that might load a library. Maybe something like ffi-library-path should be added.

The module support in Emacs has the same issue more or less; here too it is not really specified how one should located a library. As far as I can tell something like module-path exists.

I would like to see this added to Melpa rather sooner than later and I can help with that (I am a deputy maintainer). But you would also have to put in more work of course. I think it is worth it though.

This ffi could help with adaption of the dynamic module functionality a lot, even if it is only used for prototyping. But that of course assumes that it is easy to get started. We are not there yet but with more documentation and fewer sharp edges this could be quite successful.

We would also have to think about how a ffi.el that has been installed from Melpa would build ffi-module.c and how to communicate to the user that they have to install libffi. There are a few modules in Melpa, but I don't think a common pattern has emerged yet how this sort of thing should be handled. Eventually packages will appear on Melpa that use this package to interface with C, and we should avoid the same situation where everyone has to reinvent the wheel and should instead come up with best practices beforehand and document them.

So yeah, it will be a bit of work, but still: What do you think?

Ps: Please start by adding a LICENSE file and adding the permission statement to ffi.el.

[tarsius]

TL;DR With some extra effort, you might very well be responsible for a second game changer after package.el. :grinning:

[tromey]

I would also have to think about how a ffi.el that has been installed from Melpa would build ffi-module.c and how to communicate to the user that they have to install libffi

I wonder about this as well.

It would be possible to compile the FFI module's C code on first use. However, as you point out, this is more difficult because libffi is also needed. Really I was hoping to get this into the core, but I tend to doubt that will happen. Maybe some elisp could look for the usual package managers and suggest something. Or maybe most systems have libffi already anyway.

[tarsius]

It's usually better for me if you send smaller, independent patches, since the simpler ones (whitespace cleanup or whatever) I tend to land immediately.

I understand where you are coming from but I feel that this would just cause more work for both of us. I usually try to do it anyway, because it is understood as the right thing to do and I don't enjoy explaining why I didn't do it. But in this case I did not clearly see the line that splits the trivial changes from the stuff that might need more thought.

If you want me to do a second pull-request with just the trivial changes, then tell me where those end in your opinion.

Do you use the Github web interface to merge pull-requests? If so then yes I can see how in one case you can just click a button and in the other its not possible to merge at all until everything is done. But in that case I would recommend you stop doing that. Github is goddamn awful interface for doing merges or doing any kind of commits. See this if you want to read a rant about how bad it is.

And of course I am biased as the author of Magit and Forge, but it is still worth pointing out how superior that interface is. I can create a properly configured branch for a pull-request some asks me to merge by pressing b y RET. I can trivially merge just some of these commits. I can make trivial changes to someone else's commits myself and push the result to the original pull-request branch.

I recommend you give it a try, though maybe not right now. Instead, what exactly would you like me to do to make it easier for you to deal with this pull-request?

[tarsius]

It would probably be best if a module that relies on this could be loaded like a module that does not. In other words it should be loadable using (require 'MODULE), which does not only implies that it should be search for on the load-path but also that loading it like so should work even if nothing like (module-load (ffi--locate-module "ffi-module")) has run yet, i.e. the the ffi-enabled module would have to load the ffi-enabling module itself if necessary. Would that be possible?