Does not render all rows

[michaelongithub]

Hi Oli 3.0 from today.

Chrome. My standard test

    $('#tabulatortestJournal').tabulator({
        height: "600px",
        fitcolumns: false,
        columns: [
            { title: "Rownum", field: "JourID", formatter:"rownum" },
            { title: "JourID", field: "JourID" },
            { title: "JourTime", field: "JourTime" },
            { title: "Zust1", field: "Zust1" },
            { title: "Dokname", field: "Dokname" },
            { title: "Doktype", field: "Doktype" },
            { title: "JourInhaltTextOnly", field: "JourInhaltTextOnly", formatter: "textarea", width:350},
        ],
        ajaxURL: daUrl,
        ajaxConfig: "GET"
    })

JourInhaltTextOnly large text < 8000 chars as always even with only 200 rows does not render all rows, and choppy. stops rendering somewhere between 160 and 200. When scrolling all variants with pgDn, scrollbar cursor, scrollbar button Ctrl End.

Same problem behaviour. with virtualDom: false

Cheers Michael

[olifolkerd]

Hi Michael,

Which browser are you using?

Cheers

Oli

[michaelongithub]

As said: Chrome. (You mentioned not optimizing IE11 any longer)

[michaelongithub]

Hi Oli I set up a test with faker.js that demonstrates the problems. Sometimes it works, somtimes choopy, espechially when PG DN oft repeat, grid cannot draw enough. So This is a starter to test all kind of really large grids easely. I am really not a Javascript guy. But it works and you can make it better.

function ConvertDateTime(daDate) {
    var currentTime = daDate,
        hours = currentTime.getHours(),
        minutes = currentTime.getMinutes();
    seconds = currentTime.getSeconds();
    miliseconds = currentTime.getMilliseconds();

    var dateinfo = "" + minutes + ":" + seconds + ":" + miliseconds;
    return dateinfo;
}

function GenertateIpsum(i) {
    if (i % 2 == 0) {
        return faker.lorem.paragraphs(Math.round(Math.random() * 7 - 1) + 1);
    }
    else if (i % 3 == 0) {
        return faker.lorem.sentence(2, 1);
    }
    else {
        return faker.lorem.word(2);
    }
}

function GenerateData() {
    var daData = [];

    for (var i = 0; i < 1000; i++) {
        var arrayElement = {
            counter : i,
            randomName: faker.name.findName(),
            randomEmail: faker.internet.email(),
            randomDate: faker.date.between(new Date("2000-01-01"), new Date("2017-01-01")),
            randomMaterial: faker.commerce.productMaterial(),
            randomFileExt: faker.commerce.product(),
            loremIpsum: GenertateIpsum(i)            
        }
        arrayElement.randomDate = ConvertDateTime(arrayElement.randomDate);
        daData.push(arrayElement);
    };

    return daData;
}

function GenerateDataTest() {
    var dieDaten = GenerateData();

    $('#tabulatortestJournal').tabulator({
        data: dieDaten,
        height: "600px",
        fitcolumns: false,
        columns: [
            { title: "counter", field: "counter" },
            { title: "randomName", field: "randomName"},
            { title: "randomEmail", field: "randomEmail" },
            { title: "randomDate", field: "randomDate" },
            { title: "randomMaterial", field: "randomMaterial" },
            { title: "randomFileExt", field: "randomFileExt" },
            { title: "loremIpsum", field: "loremIpsum", formatter: "textarea", width: 350 }
        ]
    })
}

Cheers Michael

Or just the data, so You can test right away. TestData1000.zip

[michaelongithub]

Hi Oli played around some more with this test generator. Chrome. The grid clearly starts to be unable to render correctly and completly:

• when there are rows that are higher than the grid
• when height is largely varying from row to row (this sometimes works, but often not)

When there is constant row height: runs smoothly even with 10000 rows.

[olifolkerd]

Hey,

Thanks for the additional information,

That is really helpful, I will have a look into it further this weekend.

Cheers

Oli

[michaelongithub]

Ok, wish You success. BTW: Column resize: smaller ok, larger not adapting. You mentioned earlier that you will postpone that. Or did You fix and not working? Can put in sep. issue if You want. Cheers Michael

[olifolkerd]

Hi Michael,

Thanks for the test data it has been a great help.

The issue arises because of the use of text areas with widely varying content which results in a large variability in row height between all your rows, as Tabulator has no way of knowing the height or a row before it is rendered it calculates average row height and thus scroll bar height etc based off of the first rows height. as the data in your first row is over 440px in height and half of your rows are 20px it has cause the render margin of the virtual DOM to be absorbed very quickly resulting in visual corruption.

I am having a play with a more adaptive way to handle large variability in rows (not many users use the text area formatter with much text content so this hasnt been worked on as thoroughly as other features), i should have something in place by end of Sunday.

Cheers

Oli

[michaelongithub]

Hi Oli I am using the textarea formatter, because it seems that all the other formatters cut off cell content, if width is set. Should that not be a separate option, if text is cut or not. Or am I misunderstanding how to control show all cell content vs cut off (ellipsis). Cheers Michael

[olifolkerd]

Hi Michael,

You aren't doing anything wrong, its just most peoples data tends to have the same or roughly the same row height with each row, so this problem doesn't affect them.

Its all good im working through a solution for you at the moment.

Cheers

Oli

[olifolkerd]

Hey Michael,

I have just pushed an update to the virtual rendered on the 3.1 branch which should fix your issues.

Im hoping to release 3.1 this weekend, with yet more new and exciting features!

Cheers

Oli

[olifolkerd]

Hey @michaelongithub,

Just to let you know i have been taking a deeper dive into rendering performance for a patch update this weekend.

Have a play with the rendering_performance_improvements branch, you should find it much more responsive even on IE.

I have also added some options to speed up rendering on larger tables.

The resizableColumns option can now be set to "header" to render column resize handles in just the column headers instead of cells, which offers considerable render time improvements on tables with a lot of columns.

$("#example-table").tabulator({
    resizableColumns:"header",
});

I have also added the virtualDomBuffer to allow you to manually set size of the rendering buffer in pixels. The buffer is the amount of extra rows that are rendered beyond the start and end of the table to allow for smooth scrolling without any visible rendering. By default it automatically sets to be the same size as the table height, on large (fullscreen) tables with a lot of columns or intensive formatters this can cause a burden for the re-render of the rows in the DOM as they move so this options lets you manually set a smaller size if needed.

$("#example-table").tabulator({
    virtualDomBuffer:50, //set virtual DOM buffer to 50 px
});

Any feedback you have would be great!

Cheers

Oli :)

[michaelongithub]

Hi Oli I'll wait just until the 3.1.3 with the ColResize is out, to test it all together Cheers, Michael

[olifolkerd]

Hey,

Version 3.1.4 has just been released with these fixes in.

Cheers

Oli

[michaelongithub]

Hi Oli Tested with Chrome 59.0.3071.115 an IE11, Tabulator 3.1.4

For both: relative to their different quality (see below), your rendering seems now quite imprevious to num of rows: No difference in quality or behaviour between 1000 and 50000 rows !! (And again, only 8 cols in my test!)

Chrome is really very good now !!. VirtualDomBuffer seems to have an effect: I had a 1000 px height table. When set to 250, seems to become even smoother. With all types of vertical scroll, especially very visible effect when continously pressing on scroll arrows in scrollbar. Also col resize correct now, and very smooth, kudos.

IE 11 Has become better again but not comparable to Chrome, but acceptable imo for those who still must use it. ColResize behaviour correct now, but refreshing only after leaving drag handle. But that is acceptable.

Bugs IE11 only:

• CtrlUp/Down not correct: does not jump to Start / End, but somewhere apparently arbitrayry in between
• When clicking with mouse into grid (hand cursor appears) PgUp/Down not working more then 3 consecutive times.

Again, thanks for Your great work. Cheers Michael

[olifolkerd]

Thanks once again for the feedback Michael,

Your detailed responses are making the process of refining the Virtual DOM soo much easier, so THANK YOU :)

Great to hear that is running much smoother for you now :)

I am going to override the default actions for the home/end/page up/page down keys, and use the keybinding extension to implement these manually for a better user experience.

I will let you know when they have been pushed.

Let me know if you have any other ideas for Tabulator.

Cheers

Oli

[michaelongithub]

ok. The only feature that come to my mind right away is that the serverside infinite scroll seems gone. I cant say, that I will absolutely need it. But long term I generally would consider it a standard feature.

Otherwise with the good v-rendering and the working colresize I will start using tabulator now in the first 2 real projects. So i will come back, if I stumble over something. Otherwise the existing and planned features start to be quite comprehensive. Cheers Michael

[olifolkerd]

Hey Michael,

So i have just released 3.1.5 with the key bindings replaced.

It also has some tweaks to the renderer that should clear up the last of the randomly appearing "white space" issues.

Couldn't agree more about the serverside paginated data loading for infinite scroll, I will stick it on the road map for the 3.3 release

Once again thanks for all the help!

Cheers

Oli

[michaelongithub]

Hey Oli Same IE11 Problems with PGUp/Dwn. Regression ?

Cheers Michael

[olifolkerd]

In version 3.1.5 all keyboard navigation (page up, page down, home, end) works in my test environment,

are you sure you are using the latest version i released 20 mins ago?

Cheers

Oli

[michaelongithub]

Hmm, downloaded twice, because was unsure. Verno in source or somewhere in download would be helpful to ensure version. Try right away again

[michaelongithub]

Really seems to be the case, cleared cache, use ?v=randomno, testet also in firefox, to be sure no caching effect there, same problem. Chrome still completely OK

[olifolkerd]

hmm, very strange...

looking into it now...

[olifolkerd]

It works on every browser i try this end without issue... (firefox, chrome, edge, ie) ....hmmm

Are you clicking into the tables first?

Could you post an example of your table constructor incase something about your table configuration is interfering?

if you go to http://olifolkerd.github.io/tabulator/examples/3.1/ do the tables there work for you?

the version number is present in the bower.json and package.json files in the repo

Im sure we will get to the bottom of what is going on here, it is very strange...

[olifolkerd]

i should add that i noticed you mentioned "ctrl+up" and "ctrl+down" These dont seem to be consistent in their implementation across browsers (dosnt do anything in chrome, some us it as page up/down, some as home/end) so i havnt added them.

i have added the home key, end key, page up and page down, but not "ctrl+up" or "ctrl+down" incase that is where the confusion lies.

[michaelongithub]

Really sorry, it was the keys: I always thought home/end was for the current row.

Now, as You said: with Home/ End jumps correctly to start / End table. Also in Firefox. so in that sense tabulator is perfectly ok !! If that is your intention.

So it seems only Chrome respects Ctr Home/End in this sense.

Probably have worked too many decades in win desktop apps.

And like in https://www-archive.mozilla.org/docs/end-user/moz_shortcuts.html f.i. as example there is no Ctrl+Home at all.

But aren't we really talking about keyboard navigation inside the table? Have not found general standard for that. So I dont know what is to be expected in navigation html tables. But in Excel analogy, I would have expected that.

And http://www.guriddo.net/demo/demos/jqgrid/, which is a good and widely used grid, both key pairings are supported in all browsers. I dont know, what grids usually do. But perhaps the Excel Anlalogy is something to consider ?

But I can live well, with how it is now. Sorry again because of the confusion, Cheers, Michael

[olifolkerd]

Hey Michael,

I was taking the inspiration for the keys from browser scroll behaviour rather than table scroll behaviour.

in browsers page up/down scrolls by one window height and home/end take you to the start or end of the page, which seemed most natural to me as this is a browser based tool.

The good news is that all of these are setup through the keybindings extension, so you can really easily change them to behave how you want on your tables.

Have a look at the Keybindings Documentation for instructions on how to do this, i haven't updated the docs yet with the new actions, but the ones you will want are:

• scrollPageUp - scroll up 1x table height
• scrollPageDown - scroll down 1x table height
• scrollToStart - scroll to start of table
• scrollToEnd - scroll to end of tabl

As a rough example for you:

$("#example-table").tabulator({
    keybindings:{
        "scrollToStart" : "ctrl + 38", //ctrl + up arrow
        "scrollToEnd" : "ctrl + 40", //ctrl + down arrow
    },
});

Cheers

Oli

[michaelongithub]

Hi Oli thank you, very good solution. So I will have to learn to use npm and gulp. No alternatives to creating a tabulator.js ? But thats ok, Cheers, Michael

[olifolkerd]

All extensions are included by default. No need for a custom build

[michaelongithub]

Thanks