search

ashinn / chibi-scheme

Official chibi-scheme repository
Other
1.2k stars 142 forks source link

sexp_copy_context defined as API function but not implemented #928

Open w0utert opened 11 months ago

w0utert commented 11 months ago

Ticket #42 mentions sexp_copy_context as an API for copying Chibi contexts for fast-loading. This ticket was closed as 'fixed' with a comment that suggests this feature was implemented, and the chibi/sexp.h header file indeed contains a SEXP_API sexp sexp_copy_context (sexp ctx, sexp dst, sexp flags) declaration. The source code does not seem to contain an implementation for this function though, and trying to use it will fail linking with an undefined symbol error.

I would like to use this feature to be able to create Chibi contexts that can be used in a multithreaded environment, but which need to be initialized with a non-trivial amount of SRFI's and user code. Currently creating and initializing such a context takes several 100s of milliseconds just to load R7RS small + required SRFI's, which is bad, because the contexts are created on-the-fly, used once, and are then discarded. The Scheme code executed in such contexts takes at most a low number of milliseconds, so having ~100x as long to initialize them completely defeats the purpose of running them concurrently. I was hoping to use sexp_copy_context here, initializing a context once then copying it whenever a new thread needs to evaluate something, but it is not clear if this feature is actually present in Chibi Scheme.

ashinn commented 11 months ago

Sorry about that, this was refactored and improved by Chris Walsh in 2005c19ea.

You want to use sexp dst = sexp_gc_heap_pack(ctx, new_size_or_zero).

w0utert commented 11 months ago

This seems to work, thanks!

I do have 2 questions about sexp_gc_heap_pack though:

  1. This will re-do the packing of the source context every time I want to make copy of it, which I guess is non-trivial in terms of what it needs to do? It's relatively fast for my current contexts (~1ms) but not nearly as fast as creating child contexts & environments (which is basically free, but does not allow multithreaded use). Would it be possible to make a much faster variation of this function that just copies and doesn't do any packing?

  2. I'm confused about the new_size_or_zero argument, the API comment says that passing 0 means the packed context will have exactly enough heap space to hold what was defined in the source context and nothing more. I would expect this means allocating anything in the packed context would fail with some kind of out-of-memory error, but this does not seem to be the case (I can run code that needs allocation inside it?)

ashinn commented 1 month ago
  1. Yes, it would be possible, but 1. I'm not sure how much faster it would be, and 2. it's tricky code that I don't have time to maintain.
  2. Chibi dynamically expands the heap as needed (it's actually a chain of heap blocks).