Nub of rendering challenge

[shaunc]

Given

  {{#eg-body}}
    {{my-component value="name"}}
  {{/eg-body}}

We need to render

    {{my-component value="name"}}

In the context of the individual row. So:

1) look up contents of "eg-body" for given column 2) render contents in a custom wrapper component

What is relevant "contents of 'eg-body'? The dom nodes aren't sufficient. What we really want is the template (template string)? The question is, is that available?

[BryanCrotaz]

go and read!

[shaunc]

:) -- leave any ideas -- back in an hour

[shaunc]

(I may be able to fix this in ember-collection)

[BryanCrotaz]

I would prefer that rowheight (& anything non interactive) was done in css. Way more efficient, and user can easily override. Plus doesn't hit CSP problems

[shaunc]

hmm... well ember-collection needs rowheight to know how the rows fill out the space... will consider

[BryanCrotaz]

we can read it from css? Look at minwidth of header cell to see how. Otherwise it's a grid property. No real problem with it being that, just css is nicer if possible.

Currently body cells are 1 px too narrow from col 2 onwards. Can't see why.

[shaunc]

rowHeight -- to read from css we'll need a row to style. Maybe I can put an option in ember-collection to pass in a class name for the item wrapper.

[BryanCrotaz]

Ah yes of course. Ok leave as is then

[BryanCrotaz]

added alternate row colours and horiz align for default column layouts.

{{eg-column key="age" width=50 header="Age" footer="Age Footer" resizable=false align="center"}}

eg-body and eg-render/eg-body-cell differ in structure from equivalent header and footer. See which you think is best of the two. We should consolidate the design so it's the same everywhere.

[BryanCrotaz]

do we still need column.offset?

[shaunc]

I’ll take a look — busy today but have time in eve.

On Aug 19, 2015, at 1:51 AM, Bryan notifications@github.com wrote:

added alternate row colours and horiz align for default column layouts.

eg-body and eg-render/eg-body-cell differ in structure from equivalent header and footer. See which you think is best of the two. We should consolidate the design so it's the same everywhere.

— Reply to this email directly or view it on GitHub https://github.com/shaunc/ember-grid/issues/2#issuecomment-132393467.

[shaunc]

I have committed a buffering fix to shaunc/ember-collection -- if you pull it should scroll better; we can tweak a little to improve this further now that buffering is working. Haven't looked at our code yet today but will consider the architecture questions you brought up later on this eve.

[BryanCrotaz]

:( Rendering still broken on scroll. On slow scroll it's fine, but scrolling fast results in empty rows.

[shaunc]

:( for me too -- though I've been fooling with the parameters. The other issue is if you jerk the scroll around you can easily get doubles of cells rendered at an offset. I'm beginning to worry that the requestAnimationFrame isn't (easily) compatible with our swapping out domain nodes. Still have a few more ideas but we may have to take closer control -- e.g. insuring that during scroll elements are rendered before they seen.

[shaunc]

update -- even if I get rid of the slice in eg-body and render all the data (which hurts startup and isn't realistic for large datasets) its better but still not perfect. I'll check in this mode in a branch. If you think its ok, we might be able to approach this performance by filling out the buffer incrementally in the background.

If not, though, we'll have to take more direct control of rendering, perhaps explicitly instantiating elements in the buffer rather than letting animation simply call "rerender" for async rerender as its now done.

[shaunc]

I've push something into the "slow-load" branch.

In it I've completely disabled slicing the original content rendering (hence the "slow-load"). I wanted to see how much of the problem is unavoidable with ember-collection, as we are ways to optimize the slicing.

On the front page in the slow-load branch is our table and below it a raw "ember-collection" table for comparison. You can see the ember-collection table also has flicker; with ours its certainly worse.

In console you can see log as it renders. If you watch with a big scroll,, it does a bunch of rows, then waits then another bunch.... Even if the rows are long offscreen, they are still gumming up the queue.

If we are ok with the following, doing approximately the same with incremental slices of content wouldn't be too hard. What do you think?

I am leaning toward the thought that we may have to take over from ember-collection. If we could respond to scroll directly and move a whole chunk of rows at once (keeping in order) we'd be a lot better off than the current randomly ordered & absolutely placed, where even if you are already offscreen when you get control its hard to figure out what to do instead.

[shaunc]

@BryanCrotaz My plan is currently to derive from ember-collection and override the updateCells method. Instead of just deciding which cells will display, and allowing rerender to lazily take care of it, I will greedily get the content from the body buffers.

Do you know how to include one addon from another? I am having trouble including the component so that I can extend from it.

[BryanCrotaz]

when scroll happens, what do you get given in the args?

Don't know how to include an addon, but I'm about to add in a mixin from an addon so I'll hit the same problem

[shaunc]

Currently, when scroll happens, ember-collection marks the "cells" property as changed, which triggers re-render. In "update-cells", ember-collection reassigns which cells display what (from a map), updates their properties and re-renders them. The cell is an ember-collection place-holder, with underlying row & index as data elements, as well as style info for abs position ec uses it to render the block content:

{{~#each cells as |cell|~}}
  <div style={{{cell.style}}}>{{#unless cell.hidden}}{{yield cell.item cell.index }}{{/unless}}</div>
{{~/each~}}

So the row gets the data and index as args. However, the rerender is lazy and in fast scroll it will engage in rendering elements that have already scrolled by. (At the cost of updating visible elements which will be blank.)

Instead I'd like to override updateCells to check the actual scrollTop currently, and copy the data in for the displayed slice immediately without waiting for re-render.

(I just had one more thought, though, before i do this -- perhaps I can get a PR into ec that at least reupdates offset given current scrollTop when the async rerender happens. May help to be only one async rerender away from current scroll rather than current two -- first for rerender of EC and 2nd for rerender of content cells.)

[BryanCrotaz]

these issues do seem like generic ember-collection issues rather than specific to us

[BryanCrotaz]

our part should be limited to deciding what to render in each row

[shaunc]

They are difficult to deal with generically: generically, asking subcomponent to rerender is the best you can do. A few possible optimizations I could try to get into ember-collection:

• Possibly you could also anticipate scroll by looking at several datapoints and try to calculate velocity, but that would be tricky to do well.
• You could keep cells in an ordered ring rather than in a map. In this case, when we get a rerender in the row, we could decide -- "you asked me to rerender for row X but I know that row X has already scrolled off, and my position in the ring means I should really render row Y"

However, the former is tricky and bug-prone (and would be hard to get into official repo I think). The latter is really cheating anyway.

I will soon push another copy of my own EC -- see what you think. This will probably be the best we can do with limiting ourselves to

deciding what to render in each row

UPDATE In a nutshell, during fast scroll you want to update the whole block, and you want to do it immediately, without waiting for re-render. During slow-scroll, updating each row works fine.

[BryanCrotaz]

isn't the latter saying:

1. child cell is being reused so kick off a lazy rerender
2. when rerender comes around cell might have been repurposed so render the correct contents

Your update sounds about right. in slow scroll you're updating off screen rows. In fast scroll you're updating on screen rows so lazy is not important.

UPDATE and both of these feel like generic ember-collection issues

[shaunc]

UPDATE and both of these feel like generic ember-collection issues

Ember collection will have a hard time doing better though. In our case if we know we need an immediate update we could do a low-level grab from the body-def buffers. Generically, an async rerender is only possibility.

My plan for this now is to create a new component -- "ember-scroll-ring" which uses the layout-bin-packer to allow virtual scroll of a fixed ring of divs, and allows user to register a callback to do something to an element when it appears or disappears. Then we (in this project) can concentrate on rerendering rows as you say, but we'd be able to do it synchronously with scroll (modulo having to render the body defs in the background).

However, I am currently going to leave this, and go on to cleanup, testing, and then data-driven columns.

[BryanCrotaz]

cleanup - have you had a chance to compare your cell declaration/rendering with mine and choose which is best?

I'm looking at the best way to achieve CSP friendly styling - two addons to choose from which have a mixin to do it.

I have a dev here who's finishing up some low level Raspberry Pi stuff, then I'll put him onto writing some tests for this.

What do you think of the doc style I've started in the readme? Could you add docs for the features of the body cells that you've created (eg rowheight)?

[shaunc]

cleanup - have you had a chance to compare your cell declaration/rendering with mine and choose which is best?

I like yours. Will rearrange.

I'm looking at the best way to achieve CSP friendly styling - two addons to choose from which have a mixing to do it.

Good

I have a dev here who's finishing up some low level Raspberry Pi stuff, then I'll put him onto writing some tests for this.

Excellent! :)

What do you think of the doc style I've started in the readme? Could you add docs for the features of the body cells that you've created (eg rowheight)?

I like the style in general. The "incremental buildup" is nice at the start. It may get complex later though.

I will want to put in a column definition iterator that iterates over columns provided as data once I cleanup my stuff. We will probably want to add other stuff as well.

Perhaps e.g. where we have "add a custom footer" we drop the code for custom header?

[BryanCrotaz]

The "incremental buildup"

It's not strictly observed - I've stripped out some earlier features as I demo the next ones

columns provided as data

Not sure how we'll do the block content when providing columns as data...

Perhaps e.g. where we have "add a custom footer" we drop the code for custom header?

Don't know what you mean here

UPDATE - ah - do you mean in the doc? I deliberately put both in so that it doesn't look as though you can only have one or the other

[shaunc]

| Perhaps e.g. where we have "add a custom footer" we drop the code for custom header?

In your custom footer example, the example also includes a custom header. I didn't notice the earlier drops; I was suggesting we drop the custom header in the custom footer example.

[BryanCrotaz]

I was suggesting we drop the custom header in the custom footer example.

I deliberately put both in so that it doesn't look as though you can only have one or the other

[shaunc]

I've put in documentation for body and for overall window. (Some of it is aspirational -- have ability to call accessor in my branch; the ability to look up width, height and rowWidth are still not present.)

[shaunc]

For the moment to get going I'm in the process of writing one test per component... a little bit more than the default "it renders" but not much.

[shaunc]

vs ember-collection, we have to land https://github.com/emberjs/ember-collection/issues/26 in some form before we can stop using my branch of ember-collection. Do you want me to merge in those commits you proposed? They look like they are on the fast track anyway.

I am expanding the demo app to include a gallery showing different templates containing ember grids and their results.

+1 for restructure to fully use pods

[BryanCrotaz]

I had intended my pr to go against your repo but couldn't work out how

[BryanCrotaz]

No idea why tests are failing. I'm getting different results on first and subsequent runs!

Integration | Component | eg render/eg body cell: it renders (4, 0, 4)Rerun38 ms
Uncaught Error: you must set a resolver with `testResolver.set(resolver)`@ 6 ms
Source:     
http://localhost:7357/assets/vendor.js:39469
Died on test #2     at Object.test (http://localhost:7357/assets/test-support.js:1947:11)
    at http://localhost:7357/assets/dummy.js:1221:15
    at mod.state (http://localhost:7357/assets/vendor.js:150:29)
    at tryFinally (http://localhost:7357/assets/vendor.js:30:14)
    at requireModule (http://localhost:7357/assets/vendor.js:148:5)
    at Object.TestLoader.require (http://localhost:7357/assets/test-loader.js:29:9)
    at Object.TestLoader.loadModules (http://localhost:7357/assets/test-loader.js:21:18): Cannot read property 'set' of undefined@ 19 ms
Source:     
TypeError: Cannot read property 'set' of undefined
    at http://localhost:7357/assets/dummy.js:1226:9
    at Object.wrapper (http://localhost:7357/assets/test-support.js:1929:29)
    at Object.Test.run (http://localhost:7357/assets/test-support.js:3696:28)
    at http://localhost:7357/assets/test-support.js:3825:11
    at process (http://localhost:7357/assets/test-support.js:3384:24)
    at begin (http://localhost:7357/assets/test-support.js:3429:2)
    at http://localhost:7357/assets/test-support.js:3445:4
Uncaught TypeError: Cannot read property 'subject' of undefined@ 24 ms
Source:     
http://localhost:7357/assets/vendor.js:39469
Expected 2 assertions, but 3 were run@ 38 ms
Source:     
    at Object.test (http://localhost:7357/assets/test-support.js:1947:11)
    at http://localhost:7357/assets/dummy.js:1221:15
    at mod.state (http://localhost:7357/assets/vendor.js:150:29)
    at tryFinally (http://localhost:7357/assets/vendor.js:30:14)
    at requireModule (http://localhost:7357/assets/vendor.js:148:5)
    at Object.TestLoader.require (http://localhost:7357/assets/test-loader.js:29:9)
    at Object.TestLoader.loadModules (http://localhost:7357/assets/test-loader.js:21:18)

[shaunc]

vis ember-collection -- will get into my repo

vis tests -- I stepped back for a bit from tests. Apparently the new ember-cli release this week will resolve some problems with ember-cli-mocha and wanted to delay final decision about using it until that hits. In the meantime I have been filling out some examples in "gallery" in dummy app that won't replace tests, but will illustrate features in conjunction with documentation (and have already led me to a few bugs).