gwhitney
commented
2 years ago OK, I made a bunch of strong claims above and I thought that "showing" might be more convincing than "saying", so I built a tiny working prototype that has no factories and is based entirely on importing, but allows selective loading of operations and generic operations that depend only on other operations and will then automatically operate on any types those other operations are extended to. You can see it at https://code.studioinfinity.org/glen/picomath -- hope it is of interest.
josdejong
I wanted to separate a thread that came up a bit in the conversation in #126: the idea of having mutating operations on typed functions to perform all of the operations needed in mathjs: adding signatures, possibly replacing the implementation for signatures, re-constructing the full runtime when new types/conversions are added. I would really like to make a strong pitch for this being a good opportunity for adding such operations (I am not proposing that we disable any existing operations). To that end, let me gather the benefits of such an approach. Some of these points are covered in the prior conversation, but some are new -- I thought it might be helpful to have them all in one place.
1) It would then be possible to change to a convention in mathjs that each exported typed function in mathjs is created once, and remains constant for the lifetime of the mathjs instance. Then there is never any problem with stale references, because every reference ever created is the true reference to the object.
2) Therefore, there is no longer a need to worry about when dependencies for a function are added compared to when the function itself is added. As long as all of the functions are initialized before they are used, all will be well.
3) This in turn facilitates an organizational style in which the code file for a function, say, sum, defines a createSum, which when called on a mathjs instance, adds sum to that instance. (It could import its prerequisites and call their create functions on the instance, or it could just use a utility that creates empty versions of its prerequisites on the instance if they are not there (e.g. 'ensure(math, 'add')' and then rely on the overall import sequence to ensure that 'add' really is implemented -- I am not yet certain which will produce a better overall code layout. But with constant mutable typed functions, either is workable.)
4) It seems to me that this gets us to where module loading is the tree-shaking/dependency tracking mechanism -- only the modules you import, directly or indirectly, would be included in a bundle. (As per https://github.com/josdejong/mathjs/issues/1975)
5) And then, there's no need at all to add all of the signatures for a function at once; they can be added in any order, and it wouldn't even matter if the signatures for 'complex' happened to be added before 'number'. This allows a shift to more type-centric code organization: your create function can just add relevant signatures to to the relevant functions, without jeopardizing any previous use.
5) There should then be no difference between extending a previously existing mathjs instance and building a new one from scratch. You just call all the desired create functions with that instance as an argument, in whatever order. You won't need to figure out whether you can just call math.import or if you need to write factories. (In short, factories would be gone.)
6) Further, since typed-functions are constant but mutable, they can all be made inherently lazy, so libraries using typed-function won't need to implement any laziness. Basically, creating a typed function initially just sets the implementation to one that generates the real implementation from the signatures and replaces itself with that real implementation. So creation is super fast, and the cost of ordering the signatures and checking for conversions etc. is only paid on the first call. Moreover, between creation time and the first call, any number of additional signatures can be added, and the ordering/etc will still only happen once, at first call.
7) And now typed-functions can much more easily adapt to adding conversions/types/signatures after their first call. When one of these things happens that would potentially invalidate the generated implementation, their implementation is just set back to the "generate the real implementation" one (like setting a "dirty" flag, except there is no flag -- it's just whatever the current implementation is). For conversions (as opposed to just adding a signature), this would involve a typed instance keeping a list of all of the typed functions it owns, so that when a conversion is added, it could rapidly go through and "reset" all of the typed functions that use the destination of that conversion, and then they would lazily re-generate themselves as they are called.
I really don't see how to do the bulk of these things with immutable typed-functions, especially not the lazy adaptation to new conversions. But even just accumulating functions on a mathjs instance: if math is a mathjs instance, and then math.add for example is a constant mutable typed-function object, then when the createSum function runs on that math object, you don't have to worry if the resulting math.sum function placed on that object somehow includes "hard" references to the math.add typed-function because they will never be invalidated. (In fact, in this scheme for efficiency it's perhaps best if hard references are what's compiled in, rather than lookups through the math object, but since single property accesses are presumably quite quick we probably don't really have to worry much about which it is that's getting compiled in.)
You recently write in #126 that you have a "strong preference" to "not append to existing typed-functions. Instead, I prefer to create new instances and take a pure, functional, immutable approach." But typed-functions are in the end just arrays of regular functions. So the questions of "purity" etc. should be about whether the functions in that array are pure or not, etc. Appending to that array is orthogonal to that question: it's a matter of bookkeeping as to how you keep track of the overall behavior of that operation. That is, mutability/immutability of typed-functions is to me entirely about whether references to them can go stale/be invalidated or not. From that point of view, it seems clear that there will be fewer headaches/less overall code complexity if it's impossible for a reference to a specific typed-function to ever go stale.
I will also say referring to the discussion in #126 that the question of constant, mutable typed-functions vs non-constant, immutable typed-functions is somewhat orthogonal to the question of "Option B" vs. "Option C". Either would work with B, but yes, C relies on constancy. Since we seem to be heading to B, I don't think that has any implications about the proposal here.
Again, I am not proposing that any operations be taken away from typed-function, just that mutating operations be added, so that it would be possible to build a prototype of a factory-free, module-import-based, accumulating-behaviors version of mathjs. If that experiment doesn't work, presumably the existence of the mutating functions won't do any harm to the existing implementation. Since it might involve a modest amount of breaking behavior (not certain til I go through generating a PR for it), this point at which you are contemplating a significant revamp anyway of how a typed-function implementation arranges to call other implementations seems like a very good opportunity to give it a try. I would be happy to make such a PR, I would just want to have the design for #126 more or less settled so I could base it on that, and would like to know whether you'd like to see the "built-in laziness" implemented as a part of it from the beginning.
In short, I think constant, mutating typed-function has a very strong potential to greatly simplify the organization of mathjs without losing tree-shaking or configurability of different instances of mathjs. Thanks for taking this proposal seriously, and I am looking forward to your feedback.