Overall status.text?

[kingcons]

SICL is a worthy and ambitious project but it is hard to get a sense of the overall status or how to begin contributing. Would you consider adding a TODO/Roadmap file at the toplevel, or even a master status.text under code?

[danlentz]

I'd like to say, as a lurker, I've really found this to be an excellent resource -- even in its present form. The code is clear and very well dicumented. Other distros more focused on optimization sacrifice heavily in this regard.

I've been looking at sick-low etc wrt implementing a persistent heap. There are some bits of the GC missing, though, so I have been using informatimago's heap and mark/sweep collector instead. That said Im still very interested in SICL going forward

On Sunday, April 28, 2013, Brit Butler wrote:

SICL is a worthy and ambitious project but it is hard to get a sense of the overall status or how to begin contributing. Would you consider adding a TODO/Roadmap file at the toplevel, or even a master status.text under code?

— Reply to this email directly or view it on GitHubhttps://github.com/robert-strandh/SICL/issues/2 .

[robert-strandh]

Hello,

Thanks for your comments.

You are totally right that I need a document describing where things are and where I think they might be going.

The only excuse I have for not having done it yet is that I was not aware that there would be anyone interested in it. Also, sometimes it is not clear to me where things are going either, but of course, that would be worth saying in such a document.

I will definitely consider working on such a thing in the next few days.

[robert-strandh]

Hello,

I reworked what used to be a specification for what should be done in the future, so that it is now the beginning of a document that will evolve into some final documentation.

Intermediate versions will contain information about what I consider dead ends, promising future development, desired direction, etc. This information should be enough to serve as a roadmap.

There is a large amount of information to include, so it is going to take me a while. If there is any particular part that you are interested in, then I might be persuaded to work on that part first.

The document is located in the Specification directory, which I might decide to rename at some point in the future.

[kingcons]

@robert-strandh: These are lovely additions! Thanks very much. The main thing I still am figuring out is what the bootstrappring process looks like for a new CL. I assume you'd start by implementing a bunch of special forms/primitives and cross-compile from a host lisp. Still piecing together how you could use a module like SICL's LOOP by piggybacking on the host lisp.

[robert-strandh]

Bootstrapping is something that I have only started thinking about very recently. Maybe I will try to write a chapter about that next. One possibility is to create a target image in the host environment in the form of a vector of bytes that will then be mapped into the target system using mmap. The cross compiler would then create target objects in that image.

With respect to using SICL's LOOP macro, essentially the cross compiler HAS TO use the macros of the target system, because the host macros might (and most likely do) expand into host-specific code, perhaps in the form of function calls to host-specific routines. The cross compiler must thus be given an environment in which all important target macros are already present.

[danlentz]

are there any other documents, white papers which describe the strategy of "sliding gc"? As far as completing the gc using this strategy (for older generations) do you envision that would similarly use the sliding strategy or go to some more well known mark/sweep/copying algorithm?

In the case of using SICL to implement a persistent heap (assuming appropriate mmap code already taken care of) in addition to dealing with older generations, are there additional details you can think of offhand that remain to be completed? (a means of "pinning" objects and disabling gc with dynamic extent is one I can think of IIRC). In my use-case the intent is to be able to use compare-and-swap ops for implementation of a lock-free persistent (mmap-ed) index so changing address can be problematic. (http://github.com/danlentz/cl-ctrie) Im currently experimenting with a new version based on informatimago.com heap which is a traditional mark and sweep with no copying, which makes things a bit simpler, but I am enticed by the clarity and comprehensive nature of the VM implemented by SICL system-low etc.

There seems to be so much "finesse" involved in designing a good tagged memory architecture I am very inclined to leverage an existing one rather than develop this myself. Apart from SICL and informatimago the other obvious options seem to be boehm or mps, but those are both FLI which is less desirable. I've also had a quick look at SBCL's copying GC but it seems a bit complicated and more work involved.

Once again, I can't say enough how informative and useful this project is.

[robert-strandh]

Hi,

The sliding GC is documented in "The Garbage Collection Handbook" by Lins et al. However, it is usually written off as having too much overhead. The reason for that is that one has to build a table of mapping from old address intervals to new address intervals. However, in this book, there is an improvement that might make that overhead lower. I haven't quite understood it yet. Furthermore, I have an improvement of my own that makes it much cheaper to build and consult the table, so it might be feasible. The great advantage of the sliding collector is that the relative age of the objects is preserved because the order of allocation is preserved. You thus avoid promoting objects that might die soon.

The main reason for using a sliding collector in older generations would be if you have several generations, and you again want to have a precise idea of relative age in order to promote. But the reason for having several generations is to avoid pauses due to global stop-the-world collectors. You can fix that problem differently, namely with a concurrent collector, which is what I plan to do. A secondary reason (see below) is that it avoids fragmentation and has less space overhead than a copying collector.

My current idea is thus to have a per-thread nursery using a sliding collector and to have a global concurrent collector for the old generation. Furthermore, since I think it is feasible to represent every object with a 2-word header, I can design a global GC based on a concurrent mark-and-sweep algorithm without having to face possible fragmentation problems. Mark-and-sweep is known to have low memory overhead, and no fragmentation when objects are fixed size. Of course, objects are NOT fixed size, only their headers, so I have to think about what to do with the remainder of the objects. My hunch is that it won't matter much, and that one can probably do a combination of mark-and-sweep combined with some compaction scheme that can be run from time to time to avoid fragmentation. But I don't know yet.

Your problem of wanting fixed addresses is one that I have also considered for a long time. I had not given any thought to lock-free aspects, but I have been thinking about other things like foreign code and also avoiding rehashing when objects move. For that reason, in the old generation of SICL, objects won't move, or rather, HEADER objects won't move, and I think one can limit the moves of the remainder of the objects as well. I think perhaps using a concurrent mark-and-sweep for the header objects and an sliding collector for the remainder of the objects would be a great thing to try. You would have the advantage of fixed addresses for the global objects, while you could still avoid fragmentation by compacting the variable-size portion of the objects.

Let me give some more thought to (and read up on) what you are trying to do and get back to you about that later. Just one word: there is a technique called SLI (Speculative Lock Elision) that might make locks very cheap in the near future. Essentially, the hardware speculates that the lock won't be needed and lets multiple processors proceed concurrently, until it can no longer prove that there might be a conflict, and in this case does not commit the instructions. It is beautiful work by my friend and former colleague Jim Goodman.

Thank you for the nice words about the usefulness of SICL. Thanks to people like you, I keep up my enthusiasm to continue working on it.

Robert Strandh

[robert-strandh]

Hello,

In the case of using SICL to implement a persistent heap (assuming appropriate mmap code already taken care of) in addition to dealing with older generations, are there additional details you can think of offhand that remain to be completed? (a means of "pinning" objects and disabling gc with dynamic extent is one I can think of IIRC).

One of the things I would like to do some time in the future is to use SICL as a basis of an operating system, of which a persistent heap is of course an essential part.

Though I don't see how that is important in itself for a persistent that objects can be pinned down. But as I already mentioned, I find it valuable for other reasons. As I mention in the document in the Specification directory, I want objects in the old generation to be pinned down. And when required for (say) hash tables and similar, the objects will be promoted first so that they do not move after that.

In my use-case the intent is to be able to use compare-and-swap ops for implementation of a lock-free persistent (mmap-ed) index so changing address can be problematic. (http://github.com/danlentz/cl-ctrie) Im currently experimenting with a new version based on informatimago.com heap which is a traditional mark and sweep with no copying, which makes things a bit simpler, but I am enticed by the clarity and comprehensive nature of the VM implemented by SICL system-low etc.

Your use case is similar to that of a hash table in that you want to use the address of an object as a hash key. As I wrote above, it is for use cases like this that I want objects in the old generation to stay put. By dividing objects into a fixed-size header object and a variable-size "contents vector" I think I can simultaneously obtain that, and still avoid fragmentation, since contents vectors can not be shared and not directly reference by objects other than the "owner" header object.

I am curious though, I assumed you are working on cl-ctrie so as to have a generally useful library. But your discussion above makes it seem like you are searching for an implementation that is particularly well adapted to ctries, which suggests that you have a particular application in mind for the use of ctries rather than a generally-useful library. Is that the case?

Oh, and while I am at it, I read up on ctries. You seem to have found yourself a very nice and valuable project to work on. Congratulations!

There seems to be so much "finesse" involved in designing a good tagged memory architecture I am very inclined to leverage an existing one rather than develop this myself. Apart from SICL and informatimago the other obvious options seem to be boehm or mps, but those are both FLI which is less desirable. I've also had a quick look at SBCL's copying GC but it seems a bit complicated and more work involved.

Yes, I see. I am not sure how much infrastructure you need in order for it to be truly useful to you, since I don't know what application you have in mind. But I now have a pretty good idea of what it is that I want to do with the SICL heap, and implementing the GC should not be too complicated. So if you can tell me more about how you see SICL code as being useful to you, then I might steer my work in that direction, and I might be able to give you more precise help and/or advice.

Robert Strandh

[robert-strandh]

Oh, one more piece of information that might be useful in this context.

Occasionally, I think about issues related to multi-core architectures, threads, etc., but I can't pretend that I am very systematic about it.

Anyway, one thing that worried me was tread safety of operations such as adjusting an array or updating the class of existing instances.

When an array is adjusted, it might be the case that one thread that accesses some index will run in parallel with a thread that shrinks the array so that the index will no longer exist. By storing the size of the array in the contents vector and requiring ADJUST-ARRAY to allocate a fresh contents vector, and by forcing a thread to access the contents vector only once for each primitive operation, the problem is avoided (I hope). The result might not be the expected one, but at least the system won't crash or (worse) overwrite unrelated objects. After reading about ctries, I know think that it could be useful in some situations to use compare-and-swap to swap the contents vectors.

A similar situation happens when a class is being updated, perhaps at the same time that some other thread modifies a slot in one of its instances. Another scenario would be that the concurrent GC is scanning the contents vector while the class is being updated. For that reason, the contents vector of an instance of standard-object will contain a reference to the list of effective slots of the class as it was when the contents vector was created. Therefore the contents vector is always internally consistent.

Anyway, these are mostly ramblings as a result of my learning about ctries and as a result of starting again to think about tread safety. I am sure there are many more things to iron out in this domain.

Robert Strandh

[danlentz]

Thanks for all of your response -- I hadn't expected you would spend such time on this or I would have tried to have things in a more presentable condition to facilitate this discussion. At present, my github master does not reflect any of the recent work I have done on a "proper" heap based approach, but I am working to bring my informatimago heap branch up to the point where I can push it to github and perhaps provide some more useful basis for comparison when discussing SICL based heap.

I guess, for background, my current view on ctrie is as a 90% functional data structure, composed of a very limited number of (fixed size) node classes: snodes, tnodes, lnodes, bnodes, cnodes, and inodes. Also, there are several "sub-node" structures: RDCSS-descriptor, ref, and failed-ref. All of these, except inodes, are 100% immutable after creation (thus the particular importance of GC). This is kind of the beauty of ctrie -- nearly all the benefits of bagwell tries as well as the benefits of a mutable data structure. It was I think one of the last data structures he worked on as a successor to HAMT before his (untimely) death :(. Sadly the state of affairs wrt papers documenting ctries is terrible. They are grossly inconsistent, and it was murder trying to develop based on them. (To make matters worse, I don't know a lick of scala, so studying the existing implementation was slow-going)

The only mutable type of node are Inodes, which, further, are only mutable in a very limited way. They are essentially an extended kind of atomically stamped reference object as in Herlithy. With regards to "payload" meaning all the various types of things in the host lisp we are interested in persisting: in most cases my current approach is to serialize into octet vectors and persist these packed into unsigned-byte 64 vectors. When a situation occurs in which optimization might make a plausible difference, I implement a customized persistence -- such as for uuid type for example. This has not been too bad in terms of performance, and in my experiments with ctrie based persistent objects for example has little difficulty outperforming elephant, perec, etc. Of course that is not saying much. The diversity of heap objects already implemented in SICL offers quite a bit of potential improvent in these regards. So potentially what you are describing re: adjustable arrays, or redefined classes may indeed become relevant in that case, but at present are not a concern giving the small variety and immutability of thing in my heap.

Also, apropos of nothing (other than its what I've been working on today) I have a reasonable location-agnostic inter-ctrie addressing scheme that is based on the model of each ctrie having its own persistent heap. This is probably not important to our discussion, but I am happy to have finally arrived at a solution independent of file names, ctrie names, etc! In the end it was not difficult at all, I don't know why it took me so long to figure out .

With regards to my use-case, I am particularly interested in the heap optimized toward ctrie -- of course ctrie is a very generally useful map structure and so I see the persistent heap optimized for a fine-tuned index structure as two things that go hand in hand and can be used for all sorts of general purpose things, ie object-stores etc. I have also done some experimintation using ctrie as an ordered map rather than unordered, and in combination with an implementation of weight-balanced trees (a sketch of these is in my dstm-collections repo, although that is quite old and not ctrie integrated). This broaches the subject of transactions, for which I believe ctries have quite a lot of potential to implement extremely efficiently. But transactions, at the moment, are on the back burner as I'd first like to square away the persistence issues.

In particular, though, my goal is to develop ctrie optimized for heavily concurrent graph database. I'd really like to put it together a performant "resource-mediator" (graph db backend) for use with de.setf.resource.

I'm still cogitating about much of what you've said, but am delighted by some of the items you've mentioned wrt the future of SICL/GC.

Apologies for the disorganized and rambling nature of this email.

Regards, Dan

In the case of using SICL to implement a persistent heap (assuming appropriate mmap code already taken care of) in addition to dealing with older generations, are there additional details you can think of offhand that remain to be completed? (a means of "pinning" objects and disabling gc with dynamic extent is one I can think of IIRC).

One of the things I would like to do some time in the future is to use SICL as a basis of an operating system, of which a persistent heap is of course an essential part.

Though I don't see how that is important in itself for a persistent that objects can be pinned down. But as I already mentioned, I find it valuable for other reasons. As I mention in the document in the Specification directory, I want objects in the old generation to be pinned down. And when required for (say) hash tables and similar, the objects will be promoted first so that they do not move after that.

In my use-case the intent is to be able to use compare-and-swap ops for implementation of a lock-free persistent (mmap-ed) index so changing address can be problematic. (http://github.com/danlentz/cl-ctrie) Im currently experimenting with a new version based on informatimago.com heap which is a traditional mark and sweep with no copying, which makes things a bit simpler, but I am enticed by the clarity and comprehensive nature of the VM implemented by SICL system-low etc.

Your use case is similar to that of a hash table in that you want to use the address of an object as a hash key. As I wrote above, it is for use cases like this that I want objects in the old generation to stay put. By dividing objects into a fixed-size header object and a variable-size "contents vector" I think I can simultaneously obtain that, and still avoid fragmentation, since contents vectors can not be shared and not directly reference by objects other than the "owner" header object.

I am curious though, I assumed you are working on cl-ctrie so as to have a generally useful library. But your discussion above makes it seem like you are searching for an implementation that is particularly well adapted to ctries, which suggests that you have a particular application in mind for the use of ctries rather than a generally-useful library. Is that the case?

Oh, and while I am at it, I read up on ctries. You seem to have found yourself a very nice and valuable project to work on. Congratulations!

There seems to be so much "finesse" involved in designing a good tagged memory architecture I am very inclined to leverage an existing one rather than develop this myself. Apart from SICL and informatimago the other obvious options seem to be boehm or mps, but those are both FLI which is less desirable. I've also had a quick look at SBCL's copying GC but it seems a bit complicated and more work involved.

Yes, I see. I am not sure how much infrastructure you need in order for it to be truly useful to you, since I don't know what application you have in mind. But I now have a pretty good idea of what it is that I want to do with the SICL heap, and implementing the GC should not be too complicated. So if you can tell me more about how you see SICL code as being useful to you, then I might steer my work in that direction, and I might be able to give you more precise help and/or advice.

Robert Strandh

— Reply to this email directly or view it on GitHubhttps://github.com/robert-strandh/SICL/issues/2#issuecomment-17944389 .

[danlentz]

P.S. the two-word header approach is one of the best aspects of the sicl vm -- at least insofar as is immediately apparent to me during the course of my limited experience with the few alternatives i mentioned. This lowtag/hightag/widetag approach is just awful. I am very encouraged by your description of the use of the fixed-location header strategy in your last email. It seems so clear now that you've described it; I feel like I should have thought of that months ago... (as with so many insights after they have been explained to me, I suppose ;-)

On Wed, May 15, 2013 at 11:25 AM, robert-strandh notifications@github.comwrote:

Oh, one more piece of information that might be useful in this context.

Occasionally, I think about issues related to multi-core architectures, threads, etc., but I can't pretend that I am very systematic about it.

Anyway, one thing that worried me was tread safety of operations such as adjusting an array or updating the class of existing instances.

When an array is adjusted, it might be the case that one thread that accesses some index will run in parallel with a thread that shrinks the array so that the index will no longer exist. By storing the size of the array in the contents vector and requiring ADJUST-ARRAY to allocate a fresh contents vector, and by forcing a thread to access the contents vector only once for each primitive operation, the problem is avoided (I hope). The result might not be the expected one, but at least the system won't crash or (worse) overwrite unrelated objects. After reading about ctries, I know think that it could be useful in some situations to use compare-and-swap to swap the contents vectors.

A similar situation happens when a class is being updated, perhaps at the same time that some other thread modifies a slot in one of its instances. Another scenario would be that the concurrent GC is scanning the contents vector while the class is being updated. For that reason, the contents vector of an instance of standard-object will contain a reference to the list of effective slots of the class as it was when the contents vector was created. Therefore the contents vector is always internally consistent.

Anyway, these are mostly ramblings as a result of my learning about ctries and as a result of starting again to think about tread safety. I am sure there are many more things to iron out in this domain.

Robert Strandh

— Reply to this email directly or view it on GitHubhttps://github.com/robert-strandh/SICL/issues/2#issuecomment-17945638 .

[robert-strandh]

Hi,

On Wed, May 15, 2013 at 7:44 PM, Dan Lentz notifications@github.com wrote:

[lots of things]

Thanks for all this information. I now have a much better idea of what it is that you are trying to do.

Robert Strandh

[danlentz]

I've just been re-reading through concrete-cl backend and, although I've been most interested in just the heap/gc, the notion occurred to me that it could be very useful to make use of the other facilities as well -- in particular using the target vm (hosted, not bootstrapped; making liberal use of the host lisp where advantageous) to abstract some of the low-level detail and present a higher-level API that more closely resembled that of the other back ends I've implemented. Very simple short routines, of course, but eliminating the burden of incorporating some of the unique requirements of this backend using messy special-casing, or extending the generic backend protocol.

For example, one unique requirement of the heap backend is to support growing the size of the heap file when it runs out of storage. This is disruptive and requires careful handling because the existing heap must be unmapped, the file extended, new blocks added to the free list, and then mmapped again. So all threads need to be protected during this process as well as prepared to deal with (most-likely) a change in the sap referencing the base of the mmapped segment. It seems to me this could be more easily handled under-the-covers using the target machine facilities like a thin layer of microcode rather than introducing all kinds of locking, checks, complications etc into the ctrie backend protocol on the host lisp.

Do you think this is a reasonable idea? There are of couse other less exotic ways to handle these issues, but this would have the edge wrt simplicity, conformance and symmetry with existing code, etc. It also represents a significant benefit that SICL would provide over my current informatimago heap, which, of course, only provides storage allocation and gc.

Also, no big concern but you might want to consider renaming the various .asd files in SICL to match the name of the systems they define. Ie. instead of loop.asd sicl-loop.asd, read.asd -> sicl-read.asd, etc. This would allow the system definitions to be automatically picked up by quicklisp when the SICL directory is kept in ~/quicklisp/local-projects/ and eliminate the need for special configuration of asdf search paths or central-registry.

Dan

On Thursday, May 16, 2013, robert-strandh wrote:

Hi,

On Wed, May 15, 2013 at 7:44 PM, Dan Lentz <notifications@github.com<javascript:_e({}, 'cvml', 'notifications@github.com');>> wrote:

[lots of things]

Thanks for all this information. I now have a much better idea of what it is that you are trying to do.

Robert Strandh

— Reply to this email directly or view it on GitHubhttps://github.com/robert-strandh/SICL/issues/2#issuecomment-17981278 .

[robert-strandh]

Hello,

On Thu, May 16, 2013 at 12:36 PM, Dan Lentz notifications@github.com wrote:

Do you think this is a reasonable idea?

Wow, this is a very hard question. I still have only a vague idea of what it is that you are trying to accomplish. As I understood it, you are making a persistent heap based on your implementation of cl-ctrie. But you have not told me whether this persistent heap is going to be part of an existing CL implementation, if you are creating your own implementation, or something else. Furthermore, I don't know what kind of performance you need. The approach you are suggesting is not going to be very fast I would imagine.

Also, no big concern but you might want to consider renaming the various .asd files in SICL to match the name of the systems they define. Ie. instead of loop.asd sicl-loop.asd, read.asd -> sicl-read.asd, etc. This would allow the system definitions to be automatically picked up by quicklisp when the SICL directory is kept in ~/quicklisp/local-projects/ and eliminate the need for special configuration of asdf search paths or central-registry.

Oh, thanks. I will definitely consider that. I have been trying to read up on recent versions of ASDF, but the documentation that I have seen is incomprehensible to me.

Robert Strandh

[danlentz]

Do you think this is a reasonable idea?

Wow, this is a very hard question. I still have only a vague idea of what it is that you are trying to accomplish. As I understood it, you are making a persistent heap based on your implementation of cl-ctrie. But you have not told me whether this persistent heap is going to be part of an existing CL implementation, if you are creating your own implementation, or something else. Furthermore, I don't know what kind of performance you need. The approach you are suggesting is not going to be very fast I would imagine.

Ok, that answers my question. One danger of playing with SICL is that for any given problem one starts to see solutions in terms of embedded common-lisp vm's. Anyway w.r.t. The specific mmap / heap growth issue I was talking about I've got that worked out very clean.

I overcommit on mmap by factor of two (twice the file size). Thus when expanding I can double the size of the file without any change to the base sap. Immediately after, I mmap another segment = to the new file size, requesting it to be mapped at a specific memory sap -- namely the address that immediatly follows the last byte of the currently mapped segment. Thus the virtual address space remains contiguous and there is no change necessary to original base sap.

In the event the system can't fulfill my request to have the new segment mapped to that specific address, it's not an immediate crisis as I've just doubled in size. So I can issue a warning and there is plenty of time to prepare for an orderly move to some new address space at the time of the next doubling. (This is very unlikely though). The key is to always have the mmap overcommitted by a factor of two so the next doubling is always assured. This turns out to be a very useful technique to work with mmap.

Oh, thanks. I will definitely consider that. I have been trying to read up on recent versions of ASDF, but the documentation that I have seen is incomprehensible to me.

:) I am a bit skeptical on this matter that someone with such a deep conprehension and exhaustive vision into all facets of CL is going to be stumped by ASDF :). But for what it's worth I'd be happy to help take care of it for you so you don't have to be bothered with it. What specifically are you looking for? Would it be helpful if I forked SICL and created/updated the hierarchy of asdf files to support those pieces of SICL currently present and working? Do you have any specific requirements in this regard?

Dan

[robert-strandh]

Hello,

Congratulations to working out the mmap problem.

Though perhaps it is your turn to create a design document just like I was asked to do by redline6561 a few days ago. It is always interesting for me to see what other people are doing, and with such a document to read, I could probably give much better input than what I am able to do at the moment.

:) I am a bit skeptical on this matter that someone with such a deep conprehension and exhaustive vision into all facets of CL is going to be stumped by ASDF :).

Thanks! (I guess) :)

I think it has to do with growing impatience with bad documentation and lack of time to read source code in order to just USE a system (as opposed to hacking on it).

But for what it's worth I'd be happy to help take care of it for you so you don't have to be bothered with it.

That's a very nice offer. Thanks! I might take you up on it later. Right now I think it would be better if I try to understand ASDF in greater detail; I have problems with this in some other projects too. Furthermore, I fear that you would waste effort on parts of SICL that might be substantially modified at some later point.

I will definitely start renaming the ASDF files according to your suggestions though. Thanks again!

Robert Strandh

[danlentz]

I've been reading about slab allocators. This has potential in my current mode of very limited set of node types. https://github.com/bbu/Userland-slab-allocator seems very approachable.

On Friday, May 17, 2013, robert-strandh wrote:

Hello,

Congratulations to working out the mmap problem.

Though perhaps it is your turn to create a design document just like I was asked to do by redline6561 a few days ago. It is always interesting for me to see what other people are doing, and with such a document to read, I could probably give much better input than what I am able to do at the moment.

:) I am a bit skeptical on this matter that someone with such a deep conprehension and exhaustive vision into all facets of CL is going to be stumped by ASDF :).

Thanks! (I guess) :)

I think it has to do with growing impatience with bad documentation and lack of time to read source code in order to just USE a system (as opposed to hacking on it).

But for what it's worth I'd be happy to help take care of it for you so you don't have to be bothered with it.

That's a very nice offer. Thanks! I might take you up on it later. Right now I think it would be better if I try to understand ASDF in greater detail; I have problems with this in some other projects too. Furthermore, I fear that you would waste effort on parts of SICL that might be substantially modified at some later point.

I will definitely start renaming the ASDF files according to your suggestions though. Thanks again!

Robert Strandh

— Reply to this email directly or view it on GitHubhttps://github.com/robert-strandh/SICL/issues/2#issuecomment-18046477 .

[danlentz]

Robert,

Greetings. I just wanted to follow up to let you know I finally have gotten my github repo for cl-ctrie caught up

[danlentz]

Sorry -- got cut off . In any event, the implementation of ctrie protocol on informatimago mark and sweep heap is online and in a more presentable state. You should be able to get a pretty good picture of what is going on without slogging through much code. The relevant files are ctrie-protocol.lisp which defines the abstraction a backend must fulfill (GFs) and the cvm implementation of it is in cvm-ctrie.lisp. The files cvm-memory.lisp, and cvm-host.lisp are a few layers of facilities interfacing with the informatimago heap/GC not really anything ctrie-specific.

There are many new additions also; for one thing I finally solved how to leverage the unique capabilities of ctries to implement extremely efficient transactions together with a Herlithy lock-free dstm and pretty seamless transactional-class that is entirely transparent to the user. Just everything is atomic and you never have to think about it again. Of course it's still dstm so you can compose transactions arbitrarily using the usual atomic block approach if you need to, but it shouldn't be necessary.

Anyway, I had promised Id get that heap code online a while back and I just wanted to follow through on that.

Best regards, Dan

[danlentz]

When you're right, you're right... Check out https://groups.google.com/d/topic/sbcl-devel/2BU1sDlrT94/discussion

I don't think it obviates other approaches entirely, but it certainly seems like a game changer. Now I just gotta break out my soldering gun and bolt one of those RTM units onto my CPU... :)

[robert-strandh]

On Sat, Jun 22, 2013 at 11:33 PM, Dan Lentz notifications@github.com wrote:

When you're right, you're right... Check out https://groups.google.com/d/topic/sbcl-devel/2BU1sDlrT94/discussion

Ah, yes. I have been waiting for this day. Thanks for pointing this out.

I don't think it obviates other approaches entirely, but it certainly seems like a game changer.

Yes, absolutely. I sure hope your work will not be in vain.

Robert Strandh

[robert-strandh]

Hello,

Sorry for only now answering your previous messages with respect your repository being up to date (I have been very busy). I will

definitely continue to keep an eye on what you are doing.

Robert Strandh

[danlentz]

no worries -- sometimes it's helpful just to "think out loud". This kind of niche project, particularly in Common Lisp, does not engender much of an enthusiastic audience and affords few opportunities for such discussion -- and believe me, you've been more tolerant than most :)

On Sunday, June 23, 2013, robert-strandh wrote:

Hello,

Sorry for only now answering your previous messages with respect your repository being up to date (I have been very busy). I will

definitely continue to keep an eye on what you are doing.

Robert Strandh

— Reply to this email directly or view it on GitHubhttps://github.com/robert-strandh/SICL/issues/2#issuecomment-19869266 .