Feedback and the micro-benchmarks that made me happy and sad

[PhilllyJoy]

I've been following this on emacs-devel and am really looking forward to having dynamic modules support available in (hopefully) 25.1. Thank you for your efforts to get this implemented after so many years in feature limbo...

A couple of issues I've hit when trying to write a module:

• Executing module-load against an already loaded module is a no-op. This is very inconvenient when iterating on c code as evey change involved restarting emacs (for interactive use). Some kind of reload funcitonality (as unsafe as it might be) would make writing modules much more pleasent.
• Unless I'm doing something wrong, evaluating a function defined by am module which returns an interned nil (Qnil) crashes emacs when it tries to print the returned value.

I'd like to suggest (as https://github.com/aaptel/emacs-dynamic-module/issues/24#issuecomment-153405066 has) that it would be very useful to have low-overhead access (for read-only) to lisp strings, my experiment show the overhead currenty involved negates a lot of the potential benefit of using a module for anything that involves lots of individual string processing.

Some crude micro-benchmarking results:

• 100k calls to a no-op function provided by a module (compiled with -O2): 30ms.
• Same, but the function also gains access to a string argument with copy-string-contents: 120ms.
• And, the extra garbage involved usually triggers a GC which adds another 100ms or so (depending on all kinds of things) which brings this up to 220ms.

Measured with benchmark-run on a modified version of modt-string which avoided doing a malloc per call and returns a Qnil instead of constructing a string return value.

Although this is still considerably faster than the elisp version (yay!), it is still too slow for some concrete applications I have in mind (darn!), and for function which really need only read access to the string, and can guarantee against/deal with non-ascii contents, the 7-8x slowdown involved in the extra string copy, conversion, and GC seem entirely avoidable.

[tromey]

Could you file a separate bug, ideally with a small module reproducing the problem, for the "nil" crash?

[aaptel]

I suspect the crash is due to returning the static nil symbol that was interned in a different function. We can't do that with the frame-based memory model we have now.

I've updated all example to reflect that in one of my last commits.

[PhilllyJoy]

@aaptel, that explains it. You overlooked the modt-string module in that commit and that was the module I was playing with.

What do you think about the other two issues (reload and read-only string perf)?

[aaptel]

I will update the string example module, thanks.

I don't know if we can make module reloadable safely. I would suggest reusing the test module infrastucture for now (starting emacs in batch mode basically). Look at the modhelp.py script the module folder. Run modhelp.py init yourmodule to create the right folder structure and modhelp.py test yourmodule to compile, run a new batch emacs, load the module and run the tests in test.el. Each subcommands has a helpful -h flag.

I find this code/edit test/run loop convenient enough myself but I agree it might not be suitable for interactive stuff. You can try adding an -i/--interactive flag to modhelp to run a graphical emacs maybe?

[PhilllyJoy]

Some kind of reload funcitonality (as unsafe as it might be)

I don't know if we can make module reloadable safely.

It doesn't need to be safe - the intended use is by module writers, not users.

It could even be a compilation flag if that matters. I'm sympathetic to TDD, but I'm also quite fond of LISP and interactive development.

fwiw, clojure has some history with a similar problem, i.e unless you're careful you could end up in a dirty state when restarting a system inside a running JVM, but because you really want to avoid the overhead of a restart (in clojure's case, of the JVM), it's just a matter of mapping out the dangers and establishing coding conventions to avoid them.

I'd be quite happy occasionally crashing Emacs during module dev if, with a little more care, I can avoid the restarts altogether.

[PhilllyJoy]

@aaptel, if you're uninclined to provide an unsafe-reload func for developers I'd like to roll my but, having looked at the code, I can't see anywhere where no-clobber-on-reload is actually implemented explicitly, can you point me at the right spot in the code or is this perhaps some peculiarity of dlsym I'd need to work around?