Search Improvements

[csaftoiu]

• [x] http://elasticlunr.com/docs/index.js.html#toJSON , store index to local storage. Use cache buster on the same key to evict previous indices. This was too large to store. Ended up generating indices & zipping them from the dump_site phase.
• [x] Sort first by user-specified sort, then by searchRelevance. So if search for something, then sort by author, most relevant results for same author will go on top. Better is gmail's approach, which is to always sort by date, and have an AND booleans by default.
• [x] Search by fields, with syntax
• [x] Search by fields, with search dropdown not necessary
• [ ] Search for sequence of words? i.e. "pure intent" should get words "pure intent" side by side.
• [x] Give option for AND search? No, just always AND
• [x] Mix AND and OR in search? No, just always AND
• [x] Search date range? Or maybe just have a slider instead of pagination
• [x] Button to click "Enable Search" so it doesn't load if you use tabs

[lancesnead]

Pressing the "Click to Load Search" button just shows "Loading Search" and never allows search terms to be entered.

[csaftoiu]

Hmm, do any errors appear in the console?

[lancesnead]

Yeah a couple of errors.

JSONP: LOADING './data/data.searchIndex-part0.lz-b64.js' file:///C:/Temp/yahoo-groups-backup/eboard-tm/data/data.searchIndex-part0.lz-b64.js?cacheBuster=1475956699

Failed to load resource: net::ERR_FILE_NOT_FOUND file:///C:/Temp/yahoo-groups-backup/eboard-tm/data/data.searchIndex-part0.lz-b64.js?cacheBuster=1475956699

[csaftoiu]

Hmm, does that file indeed exist?

If not, was the site dumped without errors? It looks like it may not have rendered the search indices properly (this part of the code - https://github.com/csaftoiu/yahoo-groups-backup/blob/master/yahoo_groups_backup/subcommands/dump_site.py#L274 ). Maybe you don't have node installed? (It needs node to do it.)

[lancesnead]

Yep, that's what it seems to be. I have node.js installed, but how do I configure it to work with the script?

JSONP: LOADING 'eboard-tm2\data\data.index.js'

C:\Temp\yahoo-groups-backup\yahoo_groups_backup\subcommands\LocalJSONP.js:53 var promise = new Promise(function (resolve, reject) { ^ ReferenceError: Promise is not defined at Object.LocalJSONP (C:\Temp\yahoo-groups-backup\yahoo_groups_backup\subcommands\LocalJSONP.js:53:21) at loadMessageIndex (C:\Temp\yahoo-groups-backup\yahoo_groups_backup\subcommands\generate_search_index.js:82:21) at main (C:\Temp\yahoo-groups-backup\yahoo_groups_backup\subcommands\generate_search_index.js:43:3) at Object. (C:\Temp\yahoo-groups-backup\yahoo_groups_backup\subcommands\generate_search_index.js:23:1) at Module._compile (module.js:456:26) at Object.Module._extensions..js (module.js:474:10) at Module.load (module.js:356:32) at Function.Module._load (module.js:312:12) at Function.Module.runMain (module.js:497:10) at startup (node.js:119:16) Dumping group files... Site is ready in 'eboard-tm2'!

[lancesnead]

I was able to get search working by upgrading node to version 4.6.0.

[kdahlhaus]

I have node 4.8 installed and am getting a Max call stack size exceeded error when searching:

JSONP: LOADING 'airdrome_builders_static/data/data.messageData-27500-28000.js'
Processing 496 messages...
JSONP: LOADING 'airdrome_builders_static/data/data.messageData-28000-28500.js'
Processing 347 messages...
Done!
Stringifying...
There was an error: [RangeError: Maximum call stack size exceeded]
Dumping group files...
Site is ready in 'airdrome_builders_static'!

NOTE: Failed to render the following messages from the raw email
data. They may not have rendered properly.

[38, 50, 62, 147, 161, 177, 188, 189, 191, 223, 236, 237, 243, 247, 248, 265, 272, 282, 302, 307, 326, 329, 333, 337, 339, 341, 342, 343, 347, 359, 436, 442, 451, 456, 511,

Any ideas on how to fix this?