Open distributev opened 4 years ago
distributev
commented
4 years ago I suggested Grails framework above - Spring boot could be an option also, if Spring Boot will prove to add more value. Grails and JHipster are both built on top of Spring Boot.
Here is a link to another CRM solution built on top of Grails. From the data model perspective gr8crm is not as powerful as Moqui which is implementing UDM nevertheless gr8crm is a good gem as well.
jonesde
commented
4 years ago This would be better to discuss on the mailing list than with an issue on GitHub where very few people see it.
What you're talking about might seem like a simple technical decision but it is not. It is a complex decision with an impact in the tens of millions of dollars range (and only that small because the current user base for Moqui is fairly small). The Moqui architecture is largely based on the Apache OFBiz architecture.
This sort of question and suggestion has come up a little bit in the small world of Moqui but came up many times in OFBiz. My perspective on efforts in this direction is limited to clients I worked with and stories I heard. There were a few larger companies using OFBiz that decided to do just this: rewrite significant portions using more common tools like Spring, Grails, etc. This was often done based on demand or suggestion from developers inexperienced with ERP and the companies went for it. From lots of talk I'm aware of 2 serious efforts by different larger companies to do that, and both resulted in millions of dollars wasted in a failed effort to do so, both projects cancelled after dramatic budget overruns.
If you personally think it's a great idea and want to go for it I'd recommend doing a proof of concept effort. You could start with a small part of the functionality in Moqui, perhaps something REALLY simple like the Product/Catalog screens in POPC ERP that is mostly CrUD operations where tools like Spring and Grails are not too bad (they fall apart much more for non-CrUD sorts of functionality). Rewrite it in your tool of choice and see what it looks like.
Keep in mind as you do so that you would be building on a data model, services, and screens that have gone through hundreds of revisions to refine and improve them over the years. You would be starting with what is currently the 'finished' results of all of that. The next step for you PoC effort would be to make significant changes to it and see how that works out. This is the real world of ERP: changes and customizations are the majority of the effort involved in such systems.
Part of the reason such efforts fail is that the people involved don't understand the developer efficiency achieved with the Moqui (or OFBiz) tools. The bigger part is they don't understand that as much as possible technical details must be eliminated or simplified because the complexity of the domain for ERP systems is massive, most individuals will never be able to wrap their heads around it and manage that complexity even adequately let alone well. Anything that pushes man hours to technical details takes those man hours away from the business details, the business activities that must be supported for the system to be usable in real world business operations. Anything from fancy UI dreams to dramatic architecture changes is likely to kill projects by simple lack of delivery and budget overruns.
Developers balk and rebel and want to do things their own way, even if they have no experience with the types of systems they are being asked to build, maintain, customize, or expand. This does happen a lot with Moqui. People write Groovy or Java classes instead of using XML Actions, and they try to write their code in an object-oriented way instead of a service oriented way. In every case I've seen this has resulted in large volumes of buggy code that are a nightmare to maintain and support. There are users of Moqui doing this right now, some that I've worked with closely and are only slowly over the years changing their development practices to make their lives easier.
That said you can use the Moqui API directly in Groovy script or compiled Groovy or Java (or Scala or JPython, or whatever your tool of choice is). On the UI layer you can skip XML Screens and Forms entirely and just use React or Angular (or hopefully Vue, far more flexible) for your application and have the Moqui server just act as a REST API server.
For XML Actions, anything you can do in them can be done through the Moqui API (https://www.moqui.org/javadoc/index.html). XML Actions in Moqui are just translated to Groovy scripts and then compiled on the fly (the first time run or after a cache clear), so there is nothing in them you can't do otherwise.
All those things are possible, but that doesn't mean you should. Each of those alternatives tends to require more man hours to build and far more man hours to maintain, customize, and extend over time.
You probably don't believe me, most people don't. They try alternatives and then wonder why when I am assigned tasks I can build and test it in a fraction of the time it takes them to do it.
This doesn't mean there isn't room for improvement in Moqui Framework and other parts of Moqui, far from it. However, the general architecture and approach has been validated across hundreds of projects based on Moqui and OFBiz over the last 15 years or so (after the early years of OFBiz which started in 2001).
The great thing is you don't have to believe me, just try what I mentioned above and see what you think. The trick with that is it can't be done as a small effort. Even just the Catalog screens in POPC ERP have security (authc and authz), transaction management, search via ElasticSearch using a configured data feed from the relational database, a number of different UI elements, saved finds and scheduled screen renders for reports, etc, etc. Frameworks like Spring and Grails have plugins to help with many of these things but from past experience with others trying to do this sort of thing it requires a massive amount of time to assemble even a roughly equivalent tool set.
For Moqui itself: no, I don't believe it will never go this way. To get there someone would basically have to rewrite the existing functionality using different tools. I certainly don't have the resources to attempt such a thing, and I'm not aware of anyone using Moqui who does either. If done by very talented people that could perhaps be done for as little as one million USD, but personally I wouldn't accept such a project with that kind of budget, too likely to fail, something like ten million USD is more likely given the massive amount of code involved with such an approach.
Personally I think of it this way: with the Moqui Framework approach it is possible for a small team to build and maintain a large scope ERP application like POPC ERP. With conventional tools like Spring and Grails it would not be possible.
lightning-pro
commented
4 years ago ^_^ it seems 'this kind' of things not only happened here in China. ppl join the company in the morning and resign in the afternoon due to the xml things...
@distributev maybe you can try Axelor ?
distributev
commented
4 years ago Hi David,
I'll give you a little background on how I am aware about OfBiz and Moqui. I'm following OfBiz since before it was part of the Apache Foundation and when it came to Apache I said to myself "I have such a good smell, I knew it was something good happening there" - then I read the Moqui PDF and I said "he did it again, this guy knows his stuff very well - he is probably the best person when it comes to having real business and technical knowledge both combined" - other valuable people are either very good technically either very good on the functional level - it is very rare to find both skill combined.
I believe that you misunderstood me in the sense that you probably think that I was suggesting a complete rewrite of Moqui - this is not what I'm having in mind. It's quite the opposite - I believe there is so much valuable business knowledge throughout the UDM, USL and how these were put into production many times with OfBiz and also technical knowledge since Moqui is the second iteration after OfBiz and I believe that all this knowledge should be reused.
I'm not saying to ditch the current implementation I'm saying to open more doors for developers and for companies to enter and to give developers chances to use the skills where they are good at. Having an XML only choice might have been the single biggest reason why OFBiz and Moqui were not yet adopted mainstream despite of the very good business value they both provide.
Coming back to giving people additional facades to the existing Moqui and Mantle cores - For instance in the Spring Framework historically you could configure the app with XML (you can do the same in Moqui now - the biggest, most valuable and most complicated chunk is already in-place and it is time tested) and then in Spring developers could configure exactly the same Spring application using an alternative annotations set of Spring APIs and then developers could configure exactly the same application using yet another way of doing things which is the Java Spring Configuration APIs for configuring exactly the same app (with the benefits of having easy debugging in Java, IDE refactoring, etc) - the Spring core application is the same, the spring implementation under the cover is only one and these are only 3 different thin facades to the same core implementation of doing things in Spring.
Moqui approach could be similar - the core part in Moqui and Mantle will remain the same and all the knowledge which you described below will be kept
"Keep in mind as you do so that you would be building on a data model, services, and screens that have gone through hundreds of revisions to refine and improve them over the years. You would be starting with what is currently the 'finished' results of all of that."
No, I'm not saying to ditch this, I'm saying to extend existing UDM and USL knowledge / core implementations with additional "thin" API facades. In an ideal world after the new "thin" annotations and Java API facades will be implemented the existing Moqui applications based on XML will continue to work exactly the same as they are working now. And If I'm thinking more probably the Moqui / Mantle Java APIs are already in place the only thing which is probably missing is that the existing APIs might need to be cleaned, structurized and documented a bit more before making them public.
Even if I used the term "thin" many times above, I used this to highlight that I'm not speaking about a complete re-rewrite but I'm still aware that even this "thin" API layer is not a small effort and it might not be implemented ever. This might be an option for the 3rd iteration of refining these fine ideas after OfBiz and Moqui.
apathforward
commented
4 years ago @distributev Please try an implementation and let us know how it goes. ᐧ
On Mon, Mar 30, 2020 at 2:40 AM distributev notifications@github.com wrote:
Hi David,
I'll give you a little background on how I am aware about OfBiz and Moqui. I'm following OfBiz since before it was part of the Apache Foundation and when it came to Apache I said to myself "I have such a good smell, I knew it was something good happening there" - then I read the Moqui PDF and I said "he did it again, this guy knows his stuff very well - he is probably the best person when it comes to having real business and technical knowledge both combined" - other valuable people are either very good technically either very good on the functional level - it is very rare to find both skill combined.
I believe that you misunderstood me in the sense that you probably think that I was suggesting a complete rewrite of Moqui - this is not what I'm having in mind. It's quite the opposite - I believe there is so much valuable business knowledge throughout the UDM, USL and how these were put into production many times with OfBiz and also technical knowledge since Moqui is the second iteration after OfBiz and I believe that all this knowledge should be reused.
I'm not saying to ditch the current implementation I'm saying to open more doors for developers and for companies to enter and to give developers chances to use the skills where they are good at. Having an XML only choice might have been the single biggest reason why OFBiz and Moqui were not yet adopted mainstream despite of the very good business value they both provide.
Coming back to giving people additional facades to the existing Moqui and Mantle cores - For instance in the Spring Framework historically you could configure the app with XML (you can do the same in Moqui now - the biggest, most valuable and most complicated chunk is already in-place and it is time tested) and then in Spring developers could configure exactly the same Spring application using an alternative annotations set of Spring APIs and then developers could configure exactly the same application using yet another way of doing things which is the Java Spring Configuration APIs for configuring exactly the same app (with the benefits of having easy debugging in Java, IDE refactoring, etc) - the Spring core application is the same, the spring implementation under the cover is only one and these are only 3 different thin facades to the same core implementation of doing things in Spring.
Moqui approach could be similar - the core part in Moqui and Mantle will remain the same and all the knowledge which you described below will be kept
"Keep in mind as you do so that you would be building on a data model, services, and screens that have gone through hundreds of revisions to refine and improve them over the years. You would be starting with what is currently the 'finished' results of all of that."
No, I'm not saying to ditch this, I'm saying to extend existing UDM and USL knowledge / core implementations with additional "thin" API facades. In an ideal world after the new "thin" annotations and Java API facades will be implemented the existing Moqui applications based on XML will continue to work exactly the same as they are working now. And If I'm thinking more probably the Moqui / Mantle Java APIs are already in place the only thing which is probably missing is that the existing APIs might need to be cleaned, structurized and documented a bit more before making them public.
Even if I used the term "thin" many times above, I used this to highlight that I'm not speaking about a complete re-rewrite but I'm still aware that even this "thin" API layer is not a small effort and it might not be implemented ever. This might be an option for the 3rd iteration of refining these fine ideas after OfBiz and Moqui.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/moqui/moqui-framework/issues/407#issuecomment-605894027, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFK5KLXXRSPT3FG7B6PVOYTRKBSHXANCNFSM4LWATD2A .
jonesde
commented
4 years ago I suppose what you're writing could be interpreted many ways, and I don't know quite what you intend to communicate. There is already a comprehensive framework API that uses the fluent API approach for method call chaining and a number of other intentional design elements to be used with Groovy syntax and shortcuts.
Maybe an example with code and a summary would be helpful?
distributev
commented
4 years ago Document, promote, highlight the existence of the Moqui Java APIs. In addition to the existent Moqui sample apps show the equivalent "xml-free" way of building ERP / CRM apps in java or in groovy, kotlin using the Moqui java APIs
Ideally, try not reinventing the wheel - Take advantage of JPA annotations for persistence, bean validations for validations, Spring Boot / Grails already have annotations for controllers and services => take advantage of these and add Moqui semantics only for ERP / CRM / Accounting / UDM / USL where the added value is coming from.
Moqui is a good framework for building apps like ERP / CRM / Accounting / UDM / USL and UBPL - other frameworks are good at persistence and Java-based enterprise applications - spring boot, hibernate, grails - recognizing this will open more doors and I believe you are in pole position to bring this mainstream.
jonesde
commented
4 years ago Who is reinventing the wheel? The OFBiz Entity Engine that the Moqui Entity Facade is based on actually predates JPA. JPA was a simplification and refinement of Container Managed Entity EJBs with some ideas from other Object-Relational-Mapping tools that were all that existed. The first version of OFBiz actually used code generation based on XML files very similar to Entity EJB descriptor files, and then the issues with code generation and one object per database table become obvious quickly. The OFBiz Entity Engine and the improved variation of it in Moqui is a far more flexible tool for dealing with large, extensible data models without requiring persistence code changes.
A similar concept applies at higher levels of the framework, like with services. Internal business logic is modeled as services to benefit from the concepts of a service oriented architecture, both internally and externally. This makes is much easier to build logic that can run locally or remotely without having to do yet more object mapping. Once you say everything must be an object there is an explosion of boilerplate object mapping code that is a pain to maintain, and keeps developers thinking in object-oriented ways instead of service-oriented.
If you think that's where Moqui should go you don't understand the concepts that Moqui Framework is based on, including no object mapping or code generation for business level artifacts. Those are seriously cumbersome to work with compared to the Moqui approach.
That goes right back to my first comment. IMO Moqui would not survive if the higher level tools in the framework and UDM and above (USL, apps, etc) were written using tools that are so significantly less efficient to work with. It would also not be nearly as valuable as it would be more expensive to customize and maintain.
This is where the give it a try challenges come from... if you think that approach is so much better you can take the parts you like from UDM, USL, SimpleScreens, etc and rebuild them using your tools of choice. If it really is better, eventually people will migrate to it and Moqui as we know it will die.
Reinventing the wheel isn't the point, it's marketing. The big companies behind Java standards and commonly used corporate sponsored open source projects have huge influence on how many developers want to work with those tools. OFBiz and Moqui are from very small open source communities by comparison, and being community driven open source have no marketing budgets to help get developers excited about using this technology that allows them to do much more in much less time, and in a much less error prone way.
So yeah, developers moan and whine... but from my 20 years of experience in business system development working with hundreds of groups of developers there are always some developers who whine and moan about everything. So you go with Grails... they'll want to rewrite it using Spring Boot. Or you pick Grails and they'll want to rewrite it in Ruby and use Ruby on Rails that Grails was based on. It never ends, and likely never will, so why change fundamental technology every few years just to chase the current fads?
What's even worse is those who are most vocal about this or that tool often don't even know it well enough for a proof of concept, especially one with anywhere near equivalent framework level functionality. In one of the major failures to attempt a rewrite of OFBiz I mentioned in my first post, the loud speaking developers who convinced the company to rewrite it (using basically the JBoss SEAM Stack, which, go figure, was actually popular once), and those developers couldn't put together a proof of concept for it.
This was great for our little consulting company because we had a couple of partners and a few developers on staff who also thought that OFBiz should use something JPA/etc/etc based instead of the Entity, Service, etc stuff. I though it was great for a different reason than they did. We put a team of 6 on it for a month (paid for by the client). By the end of the month they were happy to deliver the PoC to the client and never touch it again. There was no more talk about switching to different persistence, logic, etc frameworks.
Of course Moqui does have competitors, like Odoo for example, that have very different architectures. Odoo is commercial open source and has been around for a long time (if under a different name), so they have a lot of functionality and a large user base. For purely OOTB use it competes well with Moqui and has more functionality in lots more plugins and so on. The main reason people choose Moqui over Odoo is for customizability (there are licensing differences and other reasons as well). The flexible tools and data model and service oriented architecture in Moqui make it far easier to customize and maintain.
distributev
commented
4 years ago Hi David,
I am aware of Odoo they are based in Belgium. I believe you are right in most of the things you are saying. It is a lot of marketing and it is a never ending story.
UDM, Moqui and OfBiz are all valuable tools. You are also right with "you can take the parts you like from UDM, USL, SimpleScreens, etc and rebuild them using your tools of choice." and this is possible because Moqui is a well thought system. One small thing which is difficult to argue with is that I know how to debug line by line a Java program in eclipse but I don't know how to debug an xml program in Eclipse. The same way in which you are trying to stay on what you know best, the same way the developers are also trying to program on what they know best of and this is not xml.
JHipster has an interesting approach to "So you go with Grails... they'll want to rewrite it using Spring Boot. Or you pick Grails and they'll want to rewrite it in Ruby and use Ruby on Rails that Grails was based on. It never ends, and likely never will, so why change fundamental technology every few years just to chase the current fads"
Starting from their JDL dsl they now generate Spring Boot on the backend - they could easily change to generate grails instead or any other back-end technology (including Rails, if they want to) they actually did this JHipster was initially supporting only Angular on the frontend and later on they added react and recently they added vue.js - I might try myself an exercise of parsing the Moqui xml files to generate JDL which will then be fed to JHipster - I'm curious to see how something like that could turn out. After that exercise I'll probably come to the same conclusion to which your developers came long ago "By the end of the month they were happy to deliver the PoC to the client and never touch it again. There was no more talk about switching to different persistence, logic, etc frameworks."
The work you are doing is top quality. I actually maintain for myself a short list with systems like that to use and Moqui is top of the list, second is OfBiz and the third is Odoo (for the same reasons you enumerated). I have no more arguments from my side. I did not want to argue I wanted to understand your reasons why this approach was chosen.
lightning-pro
commented
4 years ago @distributev https://github.com/axelor maybe meet your needs
acetousk
commented
1 year ago Hi David,
I am aware of Odoo they are based in Belgium. I believe you are right in most of the things you are saying. It is a lot of marketing and it is a never ending story.
UDM, Moqui and OfBiz are all valuable tools. You are also right with "you can take the parts you like from UDM, USL, SimpleScreens, etc and rebuild them using your tools of choice." and this is possible because Moqui is a well thought system. One small thing which is difficult to argue with is that I know how to debug line by line a Java program in eclipse but I don't know how to debug an xml program in Eclipse. The same way in which you are trying to stay on what you know best, the same way the developers are also trying to program on what they know best of and this is not xml.
JHipster has an interesting approach to "So you go with Grails... they'll want to rewrite it using Spring Boot. Or you pick Grails and they'll want to rewrite it in Ruby and use Ruby on Rails that Grails was based on. It never ends, and likely never will, so why change fundamental technology every few years just to chase the current fads"
Starting from their JDL dsl they now generate Spring Boot on the backend - they could easily change to generate grails instead or any other back-end technology (including Rails, if they want to) they actually did this JHipster was initially supporting only Angular on the frontend and later on they added react and recently they added vue.js - I might try myself an exercise of parsing the Moqui xml files to generate JDL which will then be fed to JHipster - I'm curious to see how something like that could turn out. After that exercise I'll probably come to the same conclusion to which your developers came long ago "By the end of the month they were happy to deliver the PoC to the client and never touch it again. There was no more talk about switching to different persistence, logic, etc frameworks."
The work you are doing is top quality. I actually maintain for myself a short list with systems like that to use and Moqui is top of the list, second is OfBiz and the third is Odoo (for the same reasons you enumerated). I have no more arguments from my side. I did not want to argue I wanted to understand your reasons why this approach was chosen.
@distributev Did you ever do anything with this? I'm intrigued about your thoughts here and it'd be interesting to talk about this idea as I have an idea similar to it.
jonesde
commented
1 year ago On a side note on this topic, part of the idea of XML Actions is to be more configuration than code, to enable less experienced people to put together certain types of logic. The idea is similar to no-/low-code platforms, though in this case no GUI editor that hides the XML has been created.
I've been looking into what a no/low code platform based on Moqui might look like, and XML Actions fit in well with XML Screens/Forms and XML Service definitions to enable a no/low code GUI that generates these files based on used input. The idea here would be to have the no/low tool only generate/edit/maintain Moqui XML files so that they can be hand tailored and run independently of the no/low code tool, requiring only moqui-framework to run (plus any other dependencies used, like mantle-udm/-usl, etc).
Hello,
I read the Moqui PDF documentation. I believe you have a very good tool for implementing business applications. If I am allowed a comparison Moqui standardized the different ways of implementing (java) business applications similarly with how Maven standardized the different ways of doing Java builds (with the big credit given to Moqui because implementing business applications is a much more complex task comparing with the task of building / compiling java projects).
If I am allowed a suggestion I believe that adding alternatives to the existing heavy XML driven approach would benefit the project. While reading the PDF documentation I was having a strange feeling like "I am asked to build a full-blown ERP system by coding in the XML programming language with some groovy scripting here and there - I know it is easy to debug a Java application but how easy is to debug an XML application? Maven took the same XML driven approach and then Gradle came to improve on that - many years ago Hibernate was driven by XML and now they moved to annotations - Spring was driven by XML and then they moved to offering annotations support and then added also a thin Java API configuration layer to make it even more developer friendly. Why with Moqui I am asked to go back to progamming with XML?"
From the business perpective Moqui XML configuration is very powerful in the sense that is allows more of a "declarative" way of implementing the desired business - with this approach developers mostly "describe the desired business result" vs. imperatively coding each and every aspect of the business solution - this is powerful concept for a business expert however most of the Java developers will stay away from committing to build a full-blown business solution in XML programming language.
Any possible gain from the following suggestions?
For instance can the existing Mantle XML entities be declared with standard JPA annotations (and eventually on top of standard JPA add a few more annotations where more UDM semantics are needed).
For instance can the existing Mantle validations be declared with standard Java bean validations annotations (and eventually on top of standard bean validations add a few more annotations where more UDM semantics are needed).
Java Bean Validation Basics - https://www.baeldung.com/javax-validation
For instance can the existing Mantle XML services be declared with standard Spring services annotations (and eventually on top of these annotations add a few more Moqui service related annotations where more Moqui semantics are needed and these would not be already available in the Spring service annotiations).
For instance would the Grails framework add any value, considering Groovy is already used and that Moqui also needs capabilities from a web framework?
Having @Annotations will already make the framework more appealing to Java programmers adding a Java API will make it even more (java types, IDE support, refactoring, easier debugging, etc).
Java-based container configuration - https://docs.spring.io/spring/docs/3.0.0.RC2/spring-framework-reference/html/ch03s11.html
How is this related with Moqui? - I believe there could be a similar MDL - Moqui Domain Language with the current semantics found the Moqui XML files (UDM entities, services, etc) where the developers will be allowed to "describe" their app and then the business app could be generated with standard technologies which developers can then further modify and enhance as per their custom needs.
www.jhipster.tech
https://start.jhipster.tech/jdl-studio/