Cyclic dependencies in standard modules

[ghost]

Assuming that numbers are intrinsic and also that numbers implement the .. -> range method (returning a range), there is currently a cyclic dependency between whatever provides the intrinsic number and the collections module.

I, personally, can't figure out a clean way to allow this cycle at run time. One bad solution would be to create the intrinsic number, then load the collections library (which uses number) and then invoke the import "collections" as collections expression inside of the intrinsic module.

Alternatively, we can just say that 1 .. 2 is a syntactic sugar for collections.range.from(1)to(2). This would mean that standardGrace needs to "use" the collections module, which I suppose is bad in terms of encapsulation of the built-in Grace-Modules?

What else could one do here?

[ghost]

Another alternative would to say that .. gives a primitive array that implements a do: method, but it is not a range.

[kjx]

On 9/03/2018, at 15:29PM, Richard Roberts notifications@github.com wrote:

Alternatively, we can just say that 1 .. 2 is a syntactic sugar for collections.range.from(1)to(2).

not syntatic sugar, but you could "define" .. as a method that did that. (and just not call it until collections are loaded)

a better idea is - given that you have hooks of some kind or other to make numbers, and strings have another hook to make ranges - and that hook can make the call to collections.

J

[ghost]

have another hook to make ranges

This is what I'm trying to avoid doing since it would entail having a range class in both the intrinsic and collections modules. If we were to go in this direction, we would ideally reduce code repetition by having the range in the collections module be a factory for the intrinsic range. If that's the case, I would argue it's better to just have all of the basic collections be intrinsic.

If the scope of this problem were just the .. operation, it's probably not too bad. However, the same problem is going to arise if a split(_) method is added strings, an asPrimeFactors(_) method is added for numbers, and so on. By treating .. as an implicit call to collections, and by treating other functions that return collection objects in the same way, we remove any dependency on collections from the intrinsic module.

This isolation I'm talking about is important because, without it, one cannot change any code that the intrinsic module depends on without causing some kind of issues at run time. Consequently, the job of implementing a dialect becomes a task in which the dialect author must understand these intrinsic-library and library-intrinsic dependencies; IMO, that goes against the purpose of the dialects.

[apblack]

I think that the presence of .. on numbers (which are intrinsic) means that ranges also need to be intrinsic. But the larger point you make is important — split(_) on Strings is now defined. ExceptionPackets have a method that returns a sequence of strings representing the stack frames that the exception just exited. Perhaps more obviously, the [...] syntax needs to compile into a sequence.

In the limit, all the collections would end up being intrinsic. But I don't think that we will get to that limit. I have some hope that we can draw the line at ranges and sequences.

At some point in the past, .. did generate a sequence (rather than a range), but that seemed silly. There was one less dependency, though.

[apblack]

I can see a way of removing the dependency on range using dependency injection. The idea is to store the class on which .. requests the from(_)to(_) method in a variable. Intrinsic would make this an error class, or perhaps sequence, if sequence is in intrinsic. Then, when range is imported, it could make a request on intrinsic to reset that variable.

This is pretty ugly, and it'm not sure that it's worth it for range, which is small and might just as well go into intrinsic. But it does provide a way of "drawing the line"

[ghost]

Hmm, I can see that what you're saying will work, but I have to agree it's not the ideal solution. What about we have a very small intrinsic collections library, which would provide a fixed-sized and a variable-sized list collection along with an iterator (we could optimize these for Moth). This could be enough to deal with range more cleanly, along with other intrinsic methods like split(_), while leaving the full collections for the standard library to implement. Is that a terrible idea?

[apblack]

It turns out to be a bad idea to have too many interfaces to collection-like things. We found that with lineups, which had a minimal interface. Students would use them and wonder why various of the collection methods were not available. That was what caused us to decide to promote square-bracket literals to have the full sequence interface.

I don’t have a better, idea, though, than to make mutable arrays, immutable sequences, and ranges, all intrinsic. That’s more than one might like, but much less than minigrace has now

[apblack]

I've recently gone through the minigrace libraries and modularized them. I wanted to minimize what was built-in. As the discussion above has shown, there are circular dependencies.

In minigrace, the built-ins are implemented in JavaScript, and exposed through a Grace module called intrinsic that provides their interface. Sequences are built-in, and sequences, rather than Sets, are used to implement things like types and mirrors: where you might expect to get a set of methods or method names, you actually get a sorted sequence.

What about the .. operation, and the indices method? These are both implemented using GraceRangeClass, which is defined by dynamically importing the collections module the first time that the .. or indices method is requested. This is conceptually circular, because collection eventually imports intrinsic, but not a problem in practice because the import is dynamic, and thus not subject to the compiler's check for circularity. [In reality, there is no check, because the built-ins are not implemented in Grace, but if they were ... we would still be able to circumvent it in this way.]

Everything works out fine just so long as no-one invokes .. or indices in the code that loads a module, or initializes any module in the cycle.

The places where I use this trick are currently:

• collections: to implement several operations on sequences (and strings), such as map, filter and ==, as well as for accessing the Range factory.
• pointBundle: to get the point factory object used to implement @
• pattern+typeBundle: for the type intersection and union classes used to implement the & and | operations on types
• errormessages: used to find close matches for a requested method name that was not present
• typeComparison: used to print a helpful message when a type-check fails.

One place where I don't use this trick is to access the findFile method used to load modules ... for obvious reasons. There is a simple version of findFile in the grace runner script (which loads the JavaScript library), and a more complete version in Grace that is loaded by the simple version.

Is using loadDynamicModule better than dependency injection? It really amounts to the same thing, but loadDynamicModule already existed, so I didn't have to create a new interface. This mechanism puts all of the cleverness (or uglyness, depending on your point of view) into the native library, which is going to be ugly anyway.

Bootstrapping is messy, but we have to do it.

[kjx]

Bootstrapping is messy, but we have to do it.

yes but it's not clear that every implementation would necessarily bootstrap itself in the same way. (If we had more than one implementation with libraries etc). On the grounds that composition is better than inheritance, I prefer a design where intrinsic objects have a minimal interface, and the rest comes in libraries. That would require more work to optimise, and for some reason feels more acceptable for collections than for e.g. numbers, strings, or booleans. On the other hand, things like Python's rich intrinsic collections with storage strategies shows the power of doing stuff in the VM, and doing it cleverly.

Really though the question is where we draw the line: just one updatable array with a minimal interface and everything else built on top of that? Or mutable and immutable versions - and then down the slippery slope with the interface? Either way, I'm not convinced special syntax for literals or primitive operations are necessary: special syntax can call into libraries (apparently Haskell will do this to "virtualise" numbers) or general syntax can invoke intrinsics in special cases (Smalltalk ByteArray and friends).

Modularising libraries is good though, and should make it easier to port things across (if we ever get there).

[KimBruce]

It appears not to directly impact the language design (perhaps aside from creating a note to implementors about possible circularities). I assume the main difficulty from a language point of view is what to include in the minimal collections library you are building that the compiler uses and other programmers can build from. As far as I can tell you've made reasonable choices.