fkleedorfer
commented
1 year ago Next steps?
In order to make the suggestions in #294, back in January, I made a prototype of the monorepo. It needed more work at the time - maybe a few days. When done, one could try it and weigh pros and cons of an actual system instead of each person's mental model of it.
It seems like a waste of time to finish the prototype unless the core team would at least consider a switch if it works well. However, if the core team is willing to consider it, I'll be more than happy to do the work, and even more delighted if others wanted to contribute to this effort.
Making the monorepo requires some changes to the existing sources (mostly to the build system, but also to package names to avoid clashes), so we would have to start from scratch with the current sources.
There will never be a good time to do this - which means the best time to do it is always: now.
beachtom
Taking up the discussion from https://github.com/IFCjs/web-ifc/issues/294
Not long ago, a new version of web-ifc-viewer was published. Thanks for doing that!!
However, it took two months to bump the web-ifc dependency , with many fixes we have been waiting for, and at least some new bugs that we are discovering only now that we can try to upgrade.
Bumping our IFC.js dependencies might be harder for us than for most as we use it in different situations (with and without GUI, and we make modifications to the model and write it). However, I suppose it is hard for everyone because so much has changed.
I know you don't owe us anything and we are glad about the work you do. However, I believe the way that the projects, builds and releases are set up constitute a tight bottleneck, preventing the project to scale the way it could, given developer interest. I am responding to the points made in #294. Let's see if they convince you. If they do, we would be more than happy to help with an alternative setup.
First off: The problems we face now:
I believe all of this is due to the main projects being maintained and built independently of each other, even though they are quite tightly coupled, and there is no reason to integrate changes to a dependency immediately, which leads to a situation in which, in large intervals, a huge release gets thrown over the fence that the downstream devs then have to chew on for quite some time. Maybe there is also the Second System Syndrome with respect to IFC.js/components going on, taking up development time. Everyone has great hopes for this... but is it ready for prime time, or will it ever be?
As stated in #294, I would suggest migrating at least web-ifc, web-ifc-three, and web-ifc-viewer, and probably also the components project into a monorepo and set up continuous releases through github actions.
As to the criticism you formulate above with respect to this approach:
@agviegas said
I believe it should be possible to set up a lerna project in such a way that you only build what you change, and pull the rest from the nightly builds repo. That would actually make it easier, not harder, to build one's own modules, for example, to fix a bug or add a feature.
I also believe that building web-ifc can be done on github through an action. I would suggest disecting web-ifc even more such that most of it can be built easily, and the hard part only needs to be touched by the developers who work on or tightly with the web assembly part.
Also, to the point of complexity: you've already got a massively complex system on your hands - this is precisely the problem. What I believe is needed is to refactor it so it consists of simpler building blocks that are easier to keep in sync with each other.
This concerns only contributors. Someone who wants to work on web-ifc, with all its complexity of building, will not be scared away by having to pull in other modules. And maybe it would be for the better for these developers, too as they might inadvertedly break dependent modules with their changes and find out about it while they work on their PR?
When I think about the speed of code change, a monorepo in my mind, makes even more sense: If you want to replace a big portion of the code, it is much easier to do so in a controlled manner as you can immediately see how your changes break other modules. You can release anytime (including consistent nightly builds, or even after each PR merge to any of the modules) so you can shorten the feedback cycle with the community. Also, such short release cycle will provide an additional incentive for developers to fork and fix bugs themselves. Bounties are great, but many just want the stuff to work - but they don't see the point of contributing if they have to wait for months to see if their PR actually fixes their problem.
@beachtom said
I think that this is also an argument for, not against a monorepo. Especially if different teams work on different modules that depend on each other, seeing each other's changes and integrating them as they are merged instead of as they are relased will help avoid duplication, divergence and finding bugs early.
Lastly, I think it would be strategically important to make this change: As far as I can tell, IFC.js has won the winner-take-all open source competition for an IFC library in JS. Everyone is rooting for you. A lot of people are ready to chime in. You want to scale. Your bounty program is proof for that. However, your project setup is a serious obstacle to growth and scaling that would otherwise take place. If you open up your build system to allow for continuous releases, you won't regret it. Let's do this!