W2UI grid is really awesome plugin.Its displaying 1,00,000 records in seconds but its crashing on google chrome while displaying 1,50,000 records from json file using W2UI grid.Please help.

[chalchis]

You should be used to load records to move the scroll

http://w2ui.com/web/demos/#!combo/combo-9

[vitmalina]

Please provide a sample file, preferably on jsFiddle where the problem is demonstrated. I am not sure I understand what you mean.

[rajb21]

{ "total": 1, "records": [ { "LEVEL":0, "IID":"TEST", "RID":"TEST", "NAME":"TEST", "OWNER":"FIELD2", "USER":"TEST", "LAST_MODIFIED_DATE":"TEST", "PN1":"", "PN2":"TEST", "STATUS":"TEST", "EXPORTED_TO":"TEST", "LAST_EXPORTED_DATE":"26-Oct-2013 21:03", "LAST_MODIFIED_DATE_1":"", "LAST_MODIFIED_DATE_2":"26-Oct-2013 03:30", "LAST_MODIFIED_DATE_3":"null", "LAST_MODIFIED_DATE_4":"null"

}, { "LEVEL":0, "IID":"TEST", "RID":"TEST", "NAME":"TEST", "OWNER":"FIELD2", "USER":"TEST", "LAST_MODIFIED_DATE":"TEST", "PN1":"", "PN2":"TEST", "STATUS":"TEST", "EXPORTED_TO":"TEST", "LAST_EXPORTED_DATE":"26-Oct-2013 21:03", "LAST_MODIFIED_DATE_1":"", "LAST_MODIFIED_DATE_2":"26-Oct-2013 03:30", "LAST_MODIFIED_DATE_3":"null", "LAST_MODIFIED_DATE_4":"null"

},

]

}

This is the sample Json file.

It will have around 2,00,000 records similar to this .

    $('#myGrid').w2grid({
        name: 'grid',
        show: {
            toolbar: true,
            footer: true,
        },
        method: 'GET', 
        columns: [{
            field: 'LEVEL',
            caption: 'Level',
            size: '30%',
            sortable: true
        }, {
            field: 'IID',
            caption: 'IID',
            size: '120px',
            hidden: true
        },

        ]
    });
    w2ui['grid'].load('abc.json');

Right now we are just fetching 2 properties for testing but it will be having 30 to 40 properties with more than 2,00,000 records.

We even tried infinite scroll option as suggested by chalchis but it is not working.

Please help.

[hansgru]

So you are trying to load in the browser a JSON file with over 1 Million data rows (not generate like http://w2ui.com/web/demos/#!combo/combo-4 but load it) ?

We even tried infinite scroll option as suggested by chalchis but it is not working.

What is not working? What errors do you encounter?

From our trials, the only way to let the user scroll/filter/show/hide/highlight quickly over more 1 Million rows is to use this this infinite scroll functionality.

Of course,the serverside implementation for the infinite scroll needs to be fast too.

[rajb21]

Thank you for the immediate reply:) When we implemented infinite scroll,initially it is displaying some set of records but as we scroll down,it is displaying blank table. We are not generating records at runtime.We want to load it from existing file. Can it be because of the json file size? Because json file with 1,50,000 records is around 5 GB in size as there are many attributes for the each record.

[hansgru]

When we implemented infinite scroll,initially it is displaying some set of records but as we scroll down,it is displaying blank table.

Was the server responding fast enough to the consequent requests ?

Can it be because of the json file size?

I think so. Let's see: you make an AJAX request that tries to load 5GB of raw json data in the Browser (in it's memory) than the raw data is transformed in a javascript "data" object structure (e.g. js array of objects) (also in memory), and then if everything still works, w2ui tries to render a part of it.

You can try with much smaller JSON files and see how is increasing the browser memory usage by increasing the JSON file size.

[rajb21]

Buffered scroll is showing same behavior for the small file also. But without it,we are able to display upto 1,00,000 records. Not more than that. If it is because of the file size,is there any other functionality of W2UI grid which we can use? Thank you for the help.

[hansgru]

If it is because of the file size,is there any other functionality of W2UI grid which we can use?

Infinite Scroll should solve your problem. You just need to implement the server part to be fast enough.

[chalchis]

my code for scroll in the grid

jQuery('#main').w2grid({
            name: 'gridusuarios',   
            autoLoad: false,
            limit: 100,
            url:'/usuarios/mod_lista',          
            header: 'Lista de usuarios del sistema',
            multiSelect: false,         
            reorderColumns: false,      
            msgRefresh: 'cargando datos...',            
            show: {
                header: true,
                footer: true,
                toolbar: true,
                toolbarReload: false,
                toolbarColumns: false,
                toolbarSearch: false,
                toolbarAdd: false,
                toolbarDelete: false,
                toolbarSave: false,
                columnHeaders: true,
                lineNumbers: true, 
                selectColumn: false,                                
                expandColumn: false
            }, 
            fixedBody: true,            
            columns: [ 
                { field: 'recid', caption: 'keygrid', size: '100px', hidden: true },
                { field: 'keyuser', caption: 'key', size: '100px', hidden: true },
                { field: 'icon', caption: '', size: '25px'},                                        
                { field: 'stus', caption: 'Estatus', size: '80px' },                    
                { field: 'username', caption: 'Matricula', size: '150px' },                            
                { field: 'nomuser', caption: 'Nombre completo', size: '600px'},
                { field: 'roluser', caption: 'Rol asignado', size: '210px' },                           
                { field: 'proceso', caption: 'Proceso', size: '100px', hidden: true },
                { field: 'facceso', caption: 'Ultimo acceso', size: '50%' },
            ],         
            onLoad: function(target, eventData)
            { 
                var serverJSON = jQuery.parseJSON(eventData.xhr.responseText);

                //console.log(serverJSON.status_respuesta);              

                if (serverJSON.status_respuesta === 'false')
                {
                    //dialogo.dialogo_error_recurso(serverJSON.mensaje_respuesta);
                    //preguntar el tipo
                    if (serverJSON.tipo_error === 'query')
                    {
                        //mostramos el mensaje del dialogo de la respuesta
                        dialogo.dialogo_error_ejecucion('<fieldset id="zonamensajes" style="max-height: 300px;">'+serverJSON.mensaje_respuesta+'</fieldset>');
                    }
                    else
                    {                    
                        //preguntar el tipo
                        if (serverJSON.tipo_error === 'accion')
                        {
                            //mostramos el mensaje del dialogo de la respuesta
                            dialogo.dialogo_error_accion('<fieldset id="zonamensajes" style="max-height: 300px;">'+serverJSON.mensaje_respuesta+'</fieldset>');
                        }
                        else
                        {
                            //mostramos el mensaje del dialogo de la respuesta
                            dialogo.dialogo_error_recurso('<fieldset id="zonamensajes" style="max-height: 300px;">'+serverJSON.mensaje_respuesta+'</fieldset>');
                        }
                    }
                }
                else
                {
                    if (serverJSON.tipo_respuesta === 'login')
                    {
                        dialogo.dialogo_error_recurso(serverJSON.mensaje_respuesta);

                        setTimeout(function(){window.location.replace('/');},1); 
                    }
                }
            },
            onKeydown: function(event) 
            {
                //si es 13 es un enter
                if (event.originalEvent.keyCode === 13)
                {
                    //emular click al boton
                    jQuery('#btn_modal_editusuario').trigger("click");
                }
            }  
        });

code in el server from url:'/usuarios/mod_lista', for example php

public function modusuarios_lista()
{
//var send for w2ui
$item_row = $this->input->post('offset', TRUE); 

//get users
$lista = $this->usuarios_modelo->get_usuarios($item_row);

//total
$total_usuarios = $this->usuarios_modelo->total_usuarios();

//json format
                $resultadoJson = array(
                                    "status_respuesta"  => "true",  
                                    "status"            => "success",//parametro del grid                                   
                                    "mensaje_respuesta" => "correcto.",
                                    "total"             => $total_usuarios,//parametro del grid                                                                     
                                    "header"            => "Listado de usuarios del sistema (".$total_usuarios.")",//parametro del gri
                                    "records"           => $datos_usuarios//parametro del grid
                                );

                echo json_encode($resultadoJson);

}

´´´
![scroll_load](https://cloud.githubusercontent.com/assets/2206648/8359017/5f0ebdf4-1b29-11e5-88ae-d8db55b6c808.gif)

[rajb21]

Thank you all for the help.

While implementing infinite scroll.it is displaying some set of records initially but it displays blank table as I scroll down. I have written same code as http://w2ui.com/web/demos/#!combo/combo-9 Just changed the url field as below: url : 'abc.json'

Its working really great for 1,00,000 records.Just need to make it work for some more data :(

Thank you in advance.

[vitmalina]

As it was already mentioned in the thread, it is unwise to load entire million records into the browser at the same time as it will consume all of the computer's memory (you even said each record was big). The example with a million records was more a proof of concept to see if it can be done. Read http://w2ui.com/web/blog/7/JavaScript-Grid-with-One-Million-Records (if you haven't already). At that much records everything starts to be slow: search, sort, etc. You should use a DB that has indexes then sort/search can be fast and you should implement server side infinite scroll.

@chalchis and @hansgru thanks for answering this post. I think I have nothing more to add. Since it is not a bug, I am closing this thread.