To reuse clojure.lang.* or to start again

[arrdem]

At present I'm playing with using Google Guava and the builder pattern for the underpinnings of Ox. Guava's datastructures are really good (more performant than Clojure's in some use cases) and this is an opportunity to not do a whole bunch more work implementing array mapped tries again.

However since they aren't based on array mapped tries, Guava maps, sets and soforth don't have remotely reasonable single entry update performance. There are many cases where this is acceptable. For instance the common idioms update-vals and update-keys involve rewriting all the entries of a map. In this "whole new structure" use case Guava maps are more performant even than transients in my benchmarking attempts. However for random access collections where new values may be inserted at any time the Guava maps really don't work.

This choice is pretty performance critical for the interpreter, since the entire immutable environment stack is build on the presumptions that: local bindings and dynamic bindings will be built and discarded, never mutated while the base bindings will never be updated and the global bindings will routinely be randomly altered via inter. So for the root bindings, a Guava map or a Clojure map would be fine. For local bindings again a Guava map will serve, but in order to make inter performant it would seem that a structurally shared map type is in order.

This propagates to the rest of the Java implementation in that I can either design around Clojure's constructor + random access update fn style, or I can take Guava's builder pattern and then deal with it style.

For some things like keywords and symbols it totally doesn't matter because there is no random access update. For the environment stack I'm working on it totally matters because random access updates are called for. Meh.

[arrdem]

I guess one intriguing option is to just copy/paste & appropriately license Clojure's perfectly functional hash mapped tries implementation and then have a long hard think about the requisite interfaces.

Ideally I'd own the world and could get some internally consistent view ala Scala of the entire space of collections. But that's not going to happen any time soon.

[danielcompton]

I'm still giving the wider issue some thought, but I'm not sure how EPL will interact with MIT.

[arrdem]

So Guava is Apache licensed, which is MIT compatible.

Note that I'm not wedded to MIT, as the limited copyleft of the EPL is attractive at some level.

[arrdem]

Ok so I think this issue is really symptoms of other things, mainly being what's Ox's relationship to Clojure's java implementation and I think the answer thereto is inspired by but entirely independent of and likely sharing none of the interfaces thereof (except IFn which is basically impossible to do any other way). All this builder pattern stuff doesn't friggin matter, it's a symptom of the clarified question.

[mikera]

@arrdem did you see this library? Might be promising...

https://github.com/GlenKPeterson/UncleJim

[arrdem]

@mikera nice! I hadn't seen that.

I chatted a little bit with Chas about this, and his comment was that to some extent trying to have a "good Java" JVM implementation means making the JVM an acceptable VM for whatever Ox turns out to be, which is putting lipstick on a pig at some level. Case in point is the fact that I would like to not have object equality running around all over the place. Having some interface representing the read/print isomorphism would be nice. The list goes on.

For convenience, I'm inclined to write about as much of Ox in Java as possible, get something that works and iterate towards something self hosting and internally consistent rather than incurring the design price of getting it all right up front. Refactoring is easy albeit boring and I think the goal as of right now should be to get something and explore not sweat specifics.

[arrdem]

Case in point, some stuff like clojure.lang.LineNumberingPushbackReader make a lot of sense and I'd have to reinvent when I could just reuse.

[mikera]

In my random hammock time on Kiss I can up with the concept of the "Abstraction" which would solve this problem. Sort of like a protocol on steroids.....

This would let you do stuff like defining an abstraction called Function

(extend-abstraction Function clojure.lang.IFn ......)

And then the language would simply work with IFns anywhere a Function is expected. Neat, tidy, and avoids the problems of getting tied to a specific implementation (unless you really want to... but I don't see why the language itself should have anything implementation specific)

Other abstractions would be things like:

• Sequence - equivalent to Clojure's sequence abstraction, but could be extended to host arrays types etc.
• Applicable - anything that can be applied at the start of an expression... includes Function as a subclass but also macros, special forms etc.
• Array - N-Dimensional array types like core.matrix, also extensible to host arrays

[mikera]

Motivation for all this was to keep the core completely host agnostic, but still provide support at a language level for efficient implementations using different underlying host constructs.

[arrdem]

Starting again using io.lucana.bifurcan.