Triggering a repaint is too hard

[flying-sheep]

Hi, I’m currently merging MathBox.js and Reveal.js for a personal talk, and hope to get a Reveal.js plugin out of it.

Unfortunately, I had to dig deep and ugly to get things to work if the slide MathBox is on is made visible too late:

var mathbox = …
  , slide = …

Reveal.addEventListener('slidechanged', function(e) {
    if (e.currentSlide === slide) {
        var ctx = tQuery.data(mathbox.world(), '_threeBoxContext')
        ctx.elementResize.callback()
    }
})

It would be nice if there was some mathbox.update() call doing this for me.

PS: the ThreeBox repo says it’s dead – Is there some MathBox 2 without it in the works or something?

PPS: What the plugin allows to do will be simply a way to hook MathBox’ Director into Reveal.js’ fragment system, so that any next- or previous-command for Reveal.js will do the right thing.

[unconed]

Yes, MathBox 2 is in the works: http://acko.net/files/dump/mathbox2/spherical.html

For my own talks, I always embedded each mathbox slideshow as an iframe, synced to the parent slideshow, it was easier and more robust. I know tQuery / ThreeBox isn't ideal, I just didn't want to roll my own system if there was something better around. However, I never ended up using the "query" part of it, and the namespace was very messy.

So I built Threestrap instead of ThreeBox to do the bootstrapping: https://github.com/unconed/threestrap

It has no opinions beyond letting you plug together simple modules and events. You should be able to pare it down to as hollow a shell around MathBox as you like with custom update behavior. Additionally, the MathBox 2 context itself is entirely threestrap-agnostic:

class Context
  constructor: (gl, scene, camera, script = []) ->

So you can use it without threestrap if you don't mind producing your own build. I just don't foresee threestrap to be so objectionable that people will want to actively avoid it.

ETA on this is a few months away though, there is a lot of work still to be done. I'm basically taking out all the magic from the primitives and exposing them as reusable, composible transforms. So instead of curve { expression: ..., color: red } you have an interval { expression: ... } (1D data buffer bound to a domain on an axis) + line { color: red } (draw bound data as line segments). And you can sandwich in more GPU powered transforms between if you like. It all gets compiled down into a vertex shader with ShaderGraph 2 (functional data flow linker for GLSL). This works today as you can see above, but there are many pieces still unimplemented.

Long story short, I no long work on MB1, but I will merge bug fixes and suggestions if they make sense.

Edit: And I do plan to provide best-effort backwards compatibility in the form of a v1 API layer that you can request. You just won't want to, because the new stuff is so much more powerful.

[flying-sheep]

i love your demos being combined with music, and many thanks for the explanation.

it doesn’t work on firefox yet, but i’m sure it will eventually (too late for my talk, though, but as said, mathbox 1 works fine)

if you don’t mind just talking in this bug tracker: i wonder why you choose to develop on chrome. it has proved to be much more unstable than firefox for edge stuff. doing a canvas/SVG project, i encountered no less than 5 breaking and 2 cosmetic chrome/webkit bugs, and only 1 cosmetical firefox bug (this one, which was both new and already fixed in the dev channel).

1. image-rendering CSS dooesn’t work in canvas 2
2. overflow-y and overflow-x with different visibilities don’t work
3. getBoundingClientRect() yields global coordinates for XHTML elements embedded in SVG
4. querySelectorAll unable to find SVG camelCase elements, such as foreignObject
5. Serializing SVG removes namespace from attributes (listed as fixed, but not yet fixed in my chromium)
6. SVG Inline content incorrectly sizes
7. clientWidth und getComputedStyle().width aren’t properly set when arriving in the DomContentLoaded-handler. circumvented with a setTimeout(100) hack. (couldn’t find a bug)

[unconed]

Heh, MathBox 2 isn't broken, Firefox is. The current stable release breaks all floating point texture uploads with gl.texSubImage2D, including the header on acko.net.

In fact my experience couldn't be more different, but then I avoid SVG and DOM as much as possible.

Firefox still can't render depth correctly when you have many CSS 3D divs, and they aren't anti-aliased either (neither is WebGL). When I was making my slideshows with CSS 3D transitions about a year ago, the layers would randomly flicker. On OS X, compositing performance is overall noticeably slower on Firefox than in Chrome, and the Firefox dev tools are still playing catch up when it comes to profiling and per frame analysis.

Compare this slideshow between Chrome and Firefox for example, it's absolutely no contest on my (somewhat older) machine, it's totally janky on Firefox. Web Audio API, same story, it was skippy until v28 for me.

Based on what I've seen, it seems Chrome is most closely positioned for the post-HTML/CSS/JS future, while Mozilla continues to deepen its marriage to them. I know which one I prefer. Heck, all the fancy new UI in Firefox 29... still full of janky, skippy animations. There's just no effort at continuity, let alone smoothness.

By avoiding the DOM entirely, I can blast 150K points out at 60fps on a phone. And it's in 3D too, unlike SVG. And look at Famo.us: they're outputting a flat list of DIVs and doing all the linear algebra in JS, outputting a single matrix per element. Because it's faster than what the browser does for you, seeing as its wrapped in layer after layer of cruft.

[flying-sheep]

i almost fully agree, but i don’t think it’s the “future”. the future of visualization, perhaps, but your stuff is highly involved: the average joe will continue to make powerpoint or at least slid.es talks, not mathbox ones. he’ll make wordpress plugins or ~ blogs, not his own blogging framework.

and even if someone does, that blog would continue to be DOM – because the DOM is useful for structured documents. luckily flexbox is coming, else we wouldn’t have useful layouting, though :)

[unconed]

CSS is already highly involved, there is nothing intuitive about it. Teaching graphical designers how to think web is already difficult. And of course MathBox is headed towards a real tool, the code / API is just a stop gap. Add in the whispering gallery around React, Angular, Famo.us, D3 and friends, and you definitely have more than visualization.

Joe Average will do what the cutting edge was doing 5 years ago, when browser support has stabilized, frameworks / libraries have caught up, and you can copy/paste your way through. That is not a useful reference point on where we need to be.

[flying-sheep]

hmm, so you think the web will head in the direction of less DOM, and not only in some niches?

because i see mathbox, a thing that’s embedded in an iframe or at least element of known aspect ratio, not something that replaces flowing text.

because as said: i fully agree that DOM-based visualizations/slide decks/games/…widgets provide more trouble than they are worth.

[unconed]

Yeah, it's annoyingly difficult for people to see beyond what's right in front of them. Again: not just about MathBox. MathBox is space and time viz, not text, but it fits in with where a lot of other things are going. I'll still have to use CSS 3D to stick bits of text on, but only as the output layer of something far more aware. Just like famous' surfaces, with their own scene graph behind it. It's faster than using native DOM nesting and native CSS 3D matrix composition.

The problem of flowing text has been massively overcomplicated in CSS by mixing it up with layout. And it still doesn't solve things people have needed since the age of QuarkXPress, like multi-column text flow, adaptive letter spacing, shaped exclusion zones, aesthetic justification, multi-line ellipses, etc. Heck, Chrome/Safari still don't apply kerning pairs. There is a lot effort invested in the world of CSS, but it moves so slowly, it will only take one smart long play to outdo it.

If you're coming at it purely from HTML/CSS/SVG, you really have an incomplete view of this. What's going on in and around the browser matters a lot. Skim e.g. this discussion on famous and see how many different concepts come together: MVC, scene graphs, observables, functional programming, layout, animation, selectors, ... These are the same things I'm dealing with in MB2, and so far I've ended up making mostly the same choices.

I think the DOM's death knell will be sounded when they finally add an API to snapshot a hidden DOM element's surface to a WebGL texture. Then we can finally use HTML for text only instead of layout, the way it was originally.

[flying-sheep]

you’re a visionary, and have much more insight to the topic than me.

i’ll keep your opiniom in mind.

[andrewdeandrade]

I think the DOM's death knell will be sounded when they finally add an API to snapshot a hidden DOM element's surface to a WebGL texture. Then we can finally use HTML for text only instead of layout, the way it was originally.

Amen. I can't wait until it's easy to have 1 window object to many document objects relationship in the browser. Then we'll finally get some of the interactivity Alan Kay claimed the web should have had from day one.

Also, this obviously isn't an API to snapshot a hidden DOM element surface to a WebGL texture, but AFAIK, this is already technically possible with something like https://github.com/trevorlinton/webkit.js/tree/master

@trevorlinton Any opinions on this?

@unconed Since CSS3 is such a pain, have you considered using famo.us as scaffolding around MathBox so you don't have to resort to the iframe? (disclaimer: I work at famo.us)

Lastly, you mentioned previously encountering difficulty with stuttering sound in some browsers. If you do encounter such issues, I'm certain that if you ping my colleague, @thealphanerd , he'd be happy to help. He knows a ton about sound on the web and is a recent post-graduate from CCRMA.

[unconed]

@malandrew Sorry but I'd prefer to lay this thread to rest. I'm happy to continue this conversation over email if you like. I didn't have issues with Web Audio, Firefox did. And I don't need Famous to make CSS 3D fast, so it seems overkill.

[eric-wieser]

For my own talks, I always embedded each mathbox slideshow as an iframe, synced to the parent slideshow, it was easier and more robust.

For anyone coming here via a google search for mathbox and reveal.js, here's how I managed to do that with reveal.js (for mathbox 2):

<section data-background-iframe="mathbox-frame.html">
    <section></section> <!-- one of these for each step in mathbox - useful for speaker notes if nothing else -->
</section>

var updateIframe = function (inds, el) {
    var bg = Reveal.getSlideBackground(inds.h);
    var iframe = bg.querySelector('iframe');
    if (!el || el == iframe) {
        iframe.contentWindow.postMessage(JSON.stringify({
            namespace: 'reveal', eventName: 'slide', value: inds.v
        }), '*');
    }
};
// update mathbox when the step changes
Reveal.addEventListener( 'slidechanged', function(e) {
    updateIframe(Reveal.getIndices(e.currentSlide));
});
// the above won't be received by iframes that aren't loaded
Reveal.addEventListener( 'ready', function() {
    Array.from(document.querySelectorAll('iframe')).forEach(function(el) {
        el.addEventListener('load', function() {
            updateIframe(Reveal.getIndices(), el);
        });
    });
});

And in your mathbox frame:

window.addEventListener('message', function(e) {
  e = JSON.parse(e.data);
  if(e.eventName == 'slide') {
    present.set('index', e.value);
  }
});