question: could Spoon be expanded to cover other/any language?

[MTyson]

Not an issue. Just a thought... Could spoon be expanded to cover other/any language? With extensive subclassing via language specific plugins, of course.

A massive amount of work, obviously. But then you'd have a universal language transform platform.

[pvojtechovsky]

I think that idea of metaprogramming is complex/abstract enough, so adding extra complexity to cover another one language is not good idea. it would make usage of spoon even harder.

But with the great tool like a spoon is, there is may be another way, how to reach your target... to use metaprogramming and to generate language specific clone of spoon model and some language independent packages automatically.

And another related idea ... for some kinds of analyzis there would be useful a more abstract model of the program. That more abstract model might be then used for transformations of code from one language to another.

By the way what is the use case behind your question? I have seen a project which implement C-language pretty printer based on the spoon model. Such project can be used to convert java code to C code...

[MTyson]

And another related idea ... for some kinds of analyzis there would be useful a more abstract model of the program

That is what I am thinking, a hierarchy that starts very general and then languages can branch at whatever level is appropriate for their syntax elements. E.g., a general ForLoop interface would apply to all languages, but an Annotation class might only apply to Java-based languages. An Annotation might descend from a more generic MetaData abstract class.

By the way what is the use case behind your question?

The ultimate purpose is actually even more advanced -- the languages are just a foundation for addressing code in a variety of frameworks, like Spring, Express or Django. Something like this, but far more flexible, powerful and expressive.

Thanks for your thoughtful response.

[pvojtechovsky]

May be look for same graph/model manipulation/transformation tool and 1) create graph in that tool using the spoon model (that graph would not be represented by nice java classes, but by abstract graph nodes of the chosen tool) 2) tranform it into another graph - the one which fits to required analyze or target language

I never did it personally and have zero experience with such graph tools. I mean something like

• graph database
• Eclipse modeling framework and related transformation
• RDF
• ...

The metadata needed by these tools might be generated by spoon metaprogramming - so not needed a huge manual effort...

@MTyson let us know if you found/develop something in that area it might be intersting for us too.

[monperrus]

Could spoon be expanded to cover other/any language? "cover other languages" may mean different things. Spoon can be extended to transform Java to another language, ie to write a transpiler, see for instance Java to C++ with https://github.com/slipperyseal/trebuchet/

[MTyson]

Keeping with the theoretical, I guess what I'm talking about is a kind of universal language (a mother language) that one could write to express higher-level logic, and then that logic be expressed in any given language at the operational level.

More practically, I'm looking at doing this for a very small and well-defined subset of functionality (like mapping openAPI definitions to functioning endpoints, creating model classes, generating CRUD database mapping code, etc), for a variety of languages and frameworks.

Maybe a lower-level approach that begins with language graphing could provide a good generic basis for plugin authors to build from -- but maybe it would be too much complexity in practice?

It certainly gets me excited to think about.

At the very least, the domain of web programming seems ripe for a meta-programming evolution. If done right, it can really speed development and improve the understandability and consistency of systems.

[pvojtechovsky]

universal language (a mother language) that one could write

There is no language which is both universal and easy to use. If one needs a easy to use language, then it always ends in a domain specific language, which is easy to use in some domain.

I think that solution is 1) let developer to express data/algorithm in any language which that developer founds usefull/easy to use for his case 2) to make AST model from that source language (if the source language is java then model can be spoon model) 3) to transform AST model of source language into universal model (the very abstract model, which is able to express all information in all languages - e.g. RDF) 4) to transform universal model into target (operational level) model and then generate target source code.

But this discussion is too abstract, so hard to say whether we speak about same things. I am sure about these things:

• It is not good idea to modify spon model to have it more universal - it would realy have a bad impact to usability of spoon.
• There will be more models in spoon environment and transformations between these models in future. But each model will be optimized for some purpose. For example Data flow controll model, ...

[monperrus]

More practically, I'm looking at doing this for a very small and well-defined subset of functionality (like mapping openAPI definitions to functioning endpoints, creating model classes, generating CRUD database mapping code, etc), for a variety of languages and frameworks.

This sounds a little bit like model transformation or graph rewriting:

https://en.wikipedia.org/wiki/Model_transformation_language

https://en.wikipedia.org/wiki/Graph_rewriting#Implementations_and_applications

[MTyson]

Thanks for the food for thought!

@pvojtechovsky, your post helped to clarify things for me. What the system in question really does/should do is: allow the construction of functional software in the desired tech stack, but comprehend, control and report on that stack's capabilities so as to provide increased control, consistency and comprehensibility.

@monperrus, thanks for pointing me at model-driven engineering, and related. That helps put words to concepts and gives me a body of work to consider. Look at this quote

"model-driven engineering technologies offer a promising approach to address the inability of third-generation languages to alleviate the complexity of platforms and express domain concepts effectively."

I have felt for a while in a vague way there is a need for a "something else" that is the real "source" behind enterprise design and architecture -- a source that holds the various, common related elements that appear again and again (data stores, schemas, endpoints, CRUD code, other boilerplate code). All the functional components are really expressions of the underlying "something else".

Your posts have helped me understand that this is essentially model-driven engineering and the "something else" is the model. Thanks.

The approach should have these characteristics:

1. Always allow direct use of the underlying technology, with a minimum/no interference from the modeling platform.
2. Rapid prototyping/build-out by using the model "prescriptively" to push changes out to implementation tech
3. Overview and heuristic capabilities by using the model "descriptively"
4. The model-driven platform should enhance and never obscure the use of the underlying tech.
5. The platform should interface cooperatively with existing model-driven artifacts like API and schema definitions.

With this in mind, I think the model in this case would be an on-demand model that exists to deliver the benefits. That is, the record of the model is the system itself, and the model is created from that and then used to support functionality. This eliminates syncing and keeps the model out of the way of coding as much as possible.

[MTyson]

The Apache Isis project does some of this, solely for Java with a specific stack.

Also of interest: ThingML Modelling -- outputs models to heterogeneous technologies.

[pvojtechovsky]

Several years ago I played with Eclipse Modeling Framework and I found it quite heavy. Many dependencies were needed and generated models depended on Eclipse a lot. All that stuff were in incubation state at that time... ... so it is probably not a good idea to try to transform Spoon model AST nodes into EMF model, to be able to do transformations. It would be probably too big in memory and inefficient - but it is just my feeling, I can be wrong.

Today I found MISO and it's JavaScript and OCL based language family Epsilon, which is good for modeling related tasks.

The interesting think is that Epsilon accesses models (including EMF and others) through a driver (=implementation of IModel interface).

Idea: One might write Epsilon IModel implementation, for Spoon model. Then we might use all the power of the Epsilon languages family and to do model transformations, validations, ... on our Spoon model directly, without necessity to convert spoon model to another - more modeling friendly data representation. The IModel is described here on page 20.

[monperrus]

For the debate: there was a long time ago, around 2006, SpoonEMF by @barais which let you to use a Spoon model in the EMF world. (note: this is not working and maintained anymore)

[MTyson]

I'm still working through everything that I've learned from this thread. Here's my current thinking:

1. Since it's true that no language (or model for that matter) is both universal and easy to use, the system in question will take a focused realm of functionality from the web development domain, including:
   • Creation of various artifacts (API endpoint handling code, model classes, persistence code, schema defs, API defs, UI Prototype)
   • The "reconciliation" of these artifacts, which is to say, the ability to generate other parts of the app based on a different part (generate API endpoints and persistence code from a schema definition, for example)
   • shorthand for common usecases
   • ability to visualize application meta-model

So far, everything appears to hinge on the "data model". I mean, the "business objects" are the target of operations and everything else is an expression of what happens to those objects.

There are all kinds of other operations, like logging in or business specific state changes, like "start test" that fall outside this scope.

However, just having a meta-model system capable of the bullets above for even just simple CRUD operations, and applicable to a variety of technology stacks via plugin would be a real achievement.

[monperrus]

FYI: https://github.com/INRIA/spoon/issues/1742

Closing the discussion here