Seems to be dead, should we do something?

skyne98 commented 6 years ago

For me it looks strange, when such "big" projects just die right away, especially when there is a big name behind it. However, there is only one thing we can do, if we still think, that idom, for example, is the right way to go for mobile and big applications: we should fork it and start working on it by ourselves.

So, as for now, I want to start a discussion:

Should we fork it and maintain it by ourselves? Are there people willing to do the maintenance work?
Is incremental dom even relevant? Maybe there was a reason for it's death?

skyne98 commented 6 years ago

As for me, right now I am working on a rendering library, which is based on idom, so, if there is an initiative to fork, I may be the person.

atomictag commented 6 years ago

I built incremental-bars based on idom and used it in several commercial projects - I am not sure why its development stalled as it's a compelling solution to many use cases. Count me in ;)

skyne98 commented 6 years ago

@atomictag, I am super happy to hear that! Based on your experience, what are the biggest pros of the idom? Memory? Performance?

atomictag commented 6 years ago

TBH for me the original main use case was to seamlessly migrate from HTML strings to DOM rendering without changing much a very Handlebars-centric legacy setup. However, it turned out speed, memory usage and overall simplicity are just great out-of-box vs anything else - although I have neither a need nor a plan to count the flops :). So I am happily using it for new stuff as well.

skyne98 commented 6 years ago

@atomictag, it's great to see, that there are still people, who prefer writing webpages practically by hand 👍. I am very glad to potentially have you as a part of the initiative. As for now, I will wait and see how many people are interested in this topic.

Meanwhile, I will both improve my library and watch this repository for updates. Maybe we will be able to hear from the author himself. If not, some time later I will do what I promised to and, maybe, we will be able to establish some kind of ecosystem.

As for now I would recommend keeping an eye on this issue for updates!

davidjamesstone commented 6 years ago

It's sad to see what's happened to incremental-dom. Strange too but, alas, I think it might have run it's course. Having said that, if there's an appetite to keep it going somehow, count me in!

I just don't really see a use case or major differentiator. Apart from reduced memory consumption due to not holding Virtual DOM trees. Maybe that's important to some applications, I don't know but for me, it's just another compile target for a sea of other available options that look kind of like elementOpen(nodeName, attributes, children) e.g. React.createElement or h.

I wrote superviews.js a template language that targets idom. I'm concentrating on hyperapp these days since idom's apparent abandonment. If you haven't seen it and you liked idom's small footprint and approach you should check it put. It's similar to idom but also provides simple state management. I've created hyperviews, the same template language but targets hyperapp.

It would be nice to hear an official line on idom's future. AFAIK, there's been nothing. For the record, there's been some more discussion in this issue. @blikblum also wrote this roundup.

paolocaminiti commented 6 years ago

For me the selling point was that idom respects the current dom state more than vdom libraries, which is it interacts better with third party code like browser extensions. While React for example would just stop rendering if the dom doesn't match it's virtual state anymore - maybe they changed this by this time, I never tested again. In the big picture of an open interoperable www of contents this certainly is a must have, but then for the daily siloed webapp you would hardly notice and we are all fine breaking the web.

My company has one project in production based on my jsonml2idom parser, and it never gave us a problem.

That said for large projects and extensibility the absence of webcomponents yet makes it hardly usable. That is once webcomponents will be as good as they were in IE5 this project would shine. And an api like this to patch the dom would even make sense as a standard, that's where elementOpen differs from a vdom.createElement. Had fun with this idea at the time.

Sad, yes. In the end this projected be doomed by not having a defined roadmap to v1 as polymer back in the days, anybody remember polymer? :)

eduardolundgren commented 6 years ago

Incremental DOM is a library that works pretty well already and do not require lots of maintenance, although it's worrisome if it gets doomed, hopefully, it won't. Google uses that internally for few projects, Google Photos if I remember correctly. Liferay, Inc. also has an entire stack of components running on top of Incremental DOM, http://github.com/metal.

@cramforce was managing Incremental DOM initially and did a good job, not sure if he still is involved, if you guys have any update would be appreciated.

skyne98 commented 6 years ago

I can easily agree with all of you! As I can see, the original authors have done a great job implementing and supporting this project for a real while.

What about use cases? I totally agree with @paolocaminiti point of view. Also, memory consumption seems like as big deal to me, because of lots of mobile websites, having hard time rendering fast, even with the latest technologies and use of AMP, or PWA. As for me exactly, right now I have a question about asynchronous usage of idom. I have see a thread discussing it, but it has abruptly ended of pull request, claiming to add this functionality. I, however, was never able to reproduce it. So is it possible? Have someone done that (open element in one tick and closing it somewhere later)? It will really help me with my very work in progress library Ink as I got stuck at using idom asynchronously.

nabendukarmakar commented 6 years ago

Incremental DOM worked great for me for one our product. We were looking for something which has the DOM diffing logic as we were rendering a large set of data and our main business logic was to apply multiple/multi-level filters on those data sets. We couldn't use React or Vue or similar libraries simply because of the size and performance.

If you would ask me, I won't let Incremental DOM die. I am thinking to fork it and then add the below priorities in mind --

Add support for components.
Making updates easier.
Do not increase the size heavily.

skyne98 commented 6 years ago

@nabendukarmakar, to be honest, I don't think idom need components. I think it was meant to be as simple and low-level as possible, for other people to build on top. Otherwise, you will introduce business logic, that anyone using it non-natively won't benefit from. As for me, as I said, I started an endeavor, to do something Moon does right now for Vue, except with idom on the backend (considering memory saving philosophy of idom) and total usage of Typescript and TSX, to allow for full type safety. Also, I had a very important idea of adding competitive multitasking into the mix, allowing Ink (as I called it) to benefit from the same features React Fiber benefits from. However, I got stuck at making idom work asynchronously. I saw a thread, but there was only "yeah, it's now possible". I wasn't able to do it. So, the project is stuck at this point. Maybe some help?

dak commented 6 years ago

I'd be interested in continuing to support development. I've used @davidjamesstone's superviews for some of my own projects and really appreciated the minimalism and performance that idom offers.

I think one of the first steps might be to just bring idom forward into 2018, as it currently has a fair bit of technical debt due to outdated dependencies. For example:

Drop the ES2015 transforms and move to using preset-env
Babel is also about to release a substantial migration to version 7

Gulp recently did release a major upgrade with version 4 Plus:

chai                                                      3.0.0  →         4.1.2 
del                                                       2.0.2  →         3.0.0 
google-closure-compiler                            20170409.0.0  →  20180402.0.0 
gulp-eslint                                               2.0.0  →         4.0.2 
gulp-replace                                              0.5.4  →         0.6.1 
gulp-sourcemaps                                           1.6.0  →         2.6.4 
karma                                                   0.13.22  →         2.0.0 
karma-chrome-launcher                                     0.2.0  →         2.2.0 
karma-firefox-launcher                                    0.1.4  →         1.1.0 
karma-mocha                                               0.2.0  →         1.3.0 
karma-rollup-preprocessor                                 2.0.1  →         5.1.1 
karma-sinon-chai                                          1.0.0  →         1.3.3 
lolex                                                     1.4.0  →         2.3.2 
mocha                                                     2.2.5  →         5.0.5 
rollup                                                   0.50.0  →        0.57.1 
rollup-plugin-babel                                       2.3.9  →         3.0.3 
rollup-plugin-multi-entry                                 1.1.0  →         2.0.2 
rollup-plugin-replace                                     1.1.1  →         2.0.0 
rollup-plugin-uglify                                      0.3.1  →         3.0.0 
sinon                                                    1.15.4  →         4.5.0 
sinon-chai                                                2.8.0  →         3.0.0 
vinyl-buffer                                              1.0.0  →         1.0.1 
vinyl-source-stream                                       1.1.0  →         2.0.0 
webcomponents.js                                          0.7.2  →        0.7.24

sparhami commented 6 years ago

I wouldn't say that Incremental DOM is dead. It is currently in use on m.youtube.com and several other things at Google. It definitely has a scope to what it does though.

Things like a component model are a bit outside of that scope (though I had hoped that web components would be a little farther along by now). You can create your own components if you so choose. I think this starts to get into the realm of where opinions start to matter and there are a lot of different choices of what to do.

Also outside of the scope are things like async or chunked rendering. For example, you might want to coordinate the rendering with other things on the page. So doing something like async rendering gets you into the business of creating a scheduler that other people can coordinate with. That space has a lot of room for opinions on how to do things as well as a lot of existing work, which isn't something that I want to expand the scope of Incremental DOM to.

Instead of implementing any chunked rendering, I would like to 1. make sure that Incremental DOM is fast enough (in real world usage) that you don't need to worry too much and 2. allow people to manage chunked work themselves at the application level (you could think of this as like an application level protocol doing things that TCP does not handle). For example, for m.youtube.com, I wrote a bit of logic to render a list of items a few at a time so that we weren't spending extra time rendering stuff way off screen before showing the user the UI.

If people find things like that useful, I can look into sharing some more assembled pieces, though they will not live in this repo.

Another example of the scope of Incremental DOM is things like attribute value sanitization is not built in. For example, Incremental DOM will let you create an anchor tag and pass it an href of "javascript:alert('foo')", and if you take the value of the href from and external source, this could be a security issue. Again, there are a lot of different ways (and opinions) on how to handle sanitizing values and passing around known safe values, which are outside of the scope of Incremental DOM, in the same way as it is outside of the scope of the setAttribute browser API.

So to be clear, we do not plan on adding a lot of new features or expanding the scope of the project. Over the course of the project, we have added some primitives to allow people to do things they couldn't before (e.g. skipNode). There might still be some work being done (e.g. we just migrated to TypeScript), but they will probably smaller changes in nature. If there are bugs, those will of course fixed.

skyne98 commented 6 years ago

@sparhami, I am very glad to hear that the project is actively used inside Google and has lots of think going for it. To make some things clear, I would say that this confusion was about the lack of issue feedback and commits in the repository.

I believe that no active project can receive no updates in some period of time due to most probably existing bugs and problems that should be fixed.

Thanks!

jimbojetlag commented 5 years ago

@skyne98 and others, please see https://github.com/google/closure-templates/issues/169 which points to "unopensourcing" SoyToIncrementalDomSrcCompiler.

This project may not be dead in Google internally, but as far as the open source community is concerned, it might be pronounced as dead. There are many accumulated unanswered issues being ignored by the team.

A quick look at github activities show that the project lead @cramforce and the rest of the team including @sparhami have migrated to the AMP project.

cramforce commented 5 years ago

I haven't been involved in a long time. The current situation is essentially that a team at Google has forked the project internally and is developing it, but unfortunately without a commitment to open-source their work.

jimbojetlag commented 5 years ago

@cramforce thanks for the update. The Google team making an internal closed-source fork is one thing, them removing the original open source code is another.

Incremental DOM is a great approach and a nice little library that has many advantages over React, including simple and fast use of server side rendered html. However, without a proper official template compiler, the library by itself is not of much use for production projects.

skyne98 commented 5 years ago

@cramforce, @jimbojetlag, it is very unfortunate to see projects being hidden from the open-source community. In my mind, there is always a question: which was the point of making them open-source in the first place? It seems like companies have different views on open-source, like, for example, Microsoft that seems to have never "unopensourced" their projects (at least big ones).

sparhami commented 5 years ago

@cramforce, @jimbojetlag, it is very unfortunate to see projects being hidden from the open-source community. In my mind, there is always a question: which was the point of making them open-source in the first place? It seems like companies have different views on open-source, like, for example, Microsoft that seems to have never "unopensourced" their projects (at least big ones).

The main problem was the closure templates implementation was not very good to begin with. This was mostly my fault. I took a lot of shortcuts to get things in a sort of working state for a subset of the Closure Templates language. One of the internal concerns with the previous implementation was that it was not able to handle some existing templates at Google and being overly restrictive on what it could do. There were also some implementation choices that were not very good, but required extensive work to do right.

A much better approach is in the works, but I'm not sure if/when that will be open sourced. Unfortunately, I think resources for supporting it outside Google are rather thin. It may make more sense for the team working on that to reconsider once things have stabilized internally more.

skyne98 commented 5 years ago

@sparhami, but why hide the source again in the first place? Why not just let people track your own internal progress, by still committing changes to the open-source repo? I don't think that the community needs extensive support and lightning-fast responses on the issues, therefore there is no need to allocate additional resources.

Just open an issue telling that "the implementation was unsatisfactory, the project is being re-imagined" and that's all - be transparent. Nevertheless, great work, great project and I bet it has its future, I hope you will consider community's issues about how this project is being treated.

jimbojetlag commented 5 years ago

Why not just let people track your own internal progress, by still committing changes to the open-source repo?

We cannot force a company to open source their internal project. It is important to distinguish between not open sourcing the rewrite version and "unopensourcing" the existing version.

The latter is harmful to the open source community, because there are a few open source projects that have hard dependency on the "unopensourced" compiler. The only way these projects can survive today is by linking to the compiler jar which is buried somewhere in a maven repo as an artifact, and even that can be pulled out without further notice.

Actions like this cause the open source community to stop investing in future Google open source libraries.

I personally believe that the idea of incremental dom is superior to React as it eliminates a lot of complexities from server side rendering. But the transparency, community support and tooling of react is not comparable with this project.

Further discussion about the "unopensourced" compiler should probably be continued in the relevant issue https://github.com/google/closure-templates/issues/169, hopefully @iteriani and @emspishak would be able to comment on this.

Maybe @sparhami could comment on what can users of idom do today given that there is no official compiler support, and that idom without a compiler support is not of much use for real world applications.

sparhami commented 5 years ago

Maybe @sparhami could comment on what can users of idom do today given that there is no official compiler support, and that idom without a compiler support is not of much use for real world applications.

I'm not sure how many people planned on using Closure Templates vs using the library with another means, but at least some set of people are using Incremental DOM either through their own template library (or one of the several in the ECOSYSTEM.md) or using a JavaScript API written on top of Incremental DOM. For example, there are usages internal to Google that use it without Closure Templates.

I think the main mistake made was in announcing / opensourcing the Closure Templates code prior to making a large enough commitment to maintain it to a level that people in the community expect.

I'd like to apologize if this has left you in a bad place or stuck on an older version of Closure Templates. As you mentioned, the best place to continue this discussion is in the closure-templates repo.

skyne98 commented 5 years ago

@sparhami, to be honest, I always treated idom as a good library for building your own templating infrastructure, therefore I haven't even heard of Closure Templates. I strongly believe that idom should be treated that way -- as a separate entity, because it is different, it has its unique way of doing things and it would be great to have something as versatile, small and understandable as idom!

jimbojetlag commented 5 years ago

I strongly believe that idom should be treated that way -- as a separate entity, because it is different, it has its unique way of doing things and it would be great to have something as versatile, small and understandable as idom!

I think you are missing a couple of points here:

The biggest advantage of Incremental DOM compared to React like approaches is easy server side rendering. And that can be even easier by sharing the template between server and client. Closure Template does exactly that, and you can find official and unofficial compilers in different languages. It is unfortunate that despite it old age, Closure Template is not known properly in the open source community.
While you can create your own, or use unofficial solutions, to compile another template to idom, as you have seen in this thread, it turned out that even for Google implementing such compiler is not trivial.

For these reasons, I believe it's important to have the official Closure Template support.

mseddon commented 5 years ago

@sparhami this project makes for excellent bedrock. I am 100% behind keeping it minimal. I see this more as OpenGL for the DOM than an opinionated framework, and this is really the value add for me, and I think a lot of other commenters here.

Too often I am fighting React, Angular and Vue because either I can't break out of a major perf bottleneck or I am forced into some contortions required by the library working against me. Honestly I think we need more libraries aimed at this level, since they facilitate future tool research.

There seems to be quite some interest in developing the approach further, perhaps a mailing list or similair could take the load and share experiences tackling particular use cases?

atomictag commented 5 years ago

I also second the comments above - especially @mseddon's view of Incremental Dom as a low-level toolkit to build on. I used Incremental Dom to make incremental-bars for my own projects and I never regretted the choice - albeit the biggest challenge is to win mindset vs the usual suspects (React, Angular, Vue...) due to its lack of community support (which I totally understand it's not necessarily a goal of this project - but many people, including myself, seem to be eager to see it getting the attention it deserves)

devinrhode2 commented 5 years ago

I don't see why the code can't be shared on github, at least to some fork that's not on github.com/google...

devinrhode2 commented 5 years ago

or hand it over to the react team to open source and maintain...

phil-hignight commented 1 year ago

Was this ever forked?

google / incremental-dom

Seems to be dead, should we do something? #352