olivernn / lunr.js

A bit like Solr, but much smaller and not as bright
http://lunrjs.com
MIT License
8.87k stars 547 forks source link

browser crash lunr.js, cause by special char in json? #458

Open hoogw opened 4 years ago

hoogw commented 4 years ago

browser crash when add doc?

click search tab, type a number, then click search button wait a minutes, browser eventually crash.

But if you try other content in this web site home page, lunr runs good, no crash. only this json data crash with lunr.

only json that crash lunr.js

image

lunr crash test page

I have enough memory, but still get out of memory crash, json array length is 150k

             ` 
                     idx = lunr(function () {

                              lunr_doc.forEach(function (doc) {

                                                   this.add(doc);

                                                 //  console.log('add lunr doc ', doc)

                                                }, this)

           })`

This same code runs good for other json, without problem.

But only one of json run into crash.

Is single or double quote in value, cause crash? for example: double quote in value:

              `  { "description": "Le Conte Avenue From 69" W/O Westwood Blvd to 69" E/O Westwood Blvd."}`

single quote in value:

                  `  { 'description': 'Le Conte Avenue From 69' W/O Westwood Blvd to 69' E/O Westwood Blvd.'}`
olivernn commented 4 years ago

Lunr doesn't parse JSON, so if there is malformed JSON the issue is before you get to load it.

I've just checked the JSON that you link to, jq is able to parse it so I think it is safe to assume that it isn't malformed.

You could try and log the ID of the document that you are adding each time you add it, that way you will be able to know which document is causing the crash.

Also, one last thing to check, you mention that you have indexed other content fine, is the other content so large? When I download that JSON it is 132MB which might be pushing Lunr and the browser to its limits.

hoogw commented 4 years ago

@olivernn very good explain, that make sense to me. I would do some experiment to testify the cause.