Add search

[AmrEldib]

Now that the site is online, it should be indexed by Google. Is the search box just a link to Google with a filter to the URL of the site, or is there a way to add search box without a server-side component?

[kashwaa]

Well, I thought I could add a JSON file that contains an array of all the posts text and their post id, it could be parsed with JS to find the post id that contains a specific keyword, but the file would be too large and will get larger with time, it will take too much time to load, there's no easy way to do this without a server-side script

[AmrEldib]

What if we broke the array into one JSON file per array item, then access them as a REST API. It's not exactly clear in my head, I need to think about this some more.

[AmrEldib]

If we index the data and store the index on the server so that each word is a folder (or JSON file) then we can call the server like this: http://egyptian-geeks.github.io/EgyptianGeeksArchive/Search/{keyword}.json Say we have two keywords, then it would two calls. Once these two files are retrieved and parsed, we can combine the two arrays by either UNION or INTERSECT depending on if the user wants OR or AND between the two keywords. The JSON file contains an array of all the posts that contain that keyword. It can storing a very simple file. Just an array of all the post IDs. That would be the smallest array possible. Then we combine the two arrays to figure out the result. In another call, we can get the preview for the top ten posts, then the second ten, and so on.

[mtayseer]

The approach described above has many disadvantages

1. We need to do the tokenization ourselves, e.g. "file" & "files" need to point to the same posts.
2. We will generate a large number of keyword files
3. If the user is searching for more than one keyword, e.g. ruby on rails, we will have to do the query analysis ourselves.

In other words, we are implementing our own search indexer, which is way out of scope for us.

I believe the simplest thing to do is using Google Search box

[AmrEldib]

Agree

[kashwaa]

I agree as well, we also will need to evade indexing common words such as "the", "a", "in" etc... I tried using a key value store from parse.com to return post id of posts containing search words, the problem now seems to be that parse.com doesn't contain a "LIKE" of "Contains" query

[kashwaa]

the key value store solution seems to be working fine, I just finished uploading and testing data into parse.com, results could be obtained using the script:

Parse.initialize("eyCdVi2Jf65mFDCFYt0D7Swuzmek4ciWl1tz9fxs", "SWLaqfJLCui6BKToomZL10ioZrIG6AS3dRAOOkMP"); var EgyGeeks = Parse.Object.extend("EgyGeeks"); var egygeeks = new EgyGeeks(); var query = new Parse.Query(EgyGeeks); query.containsAll("Data", ["search","words"]).select("PostId"); query.limit=1000; query.find({ success: function(object) { // Successfully retrieved the object. }, error: function(error) { alert("Error: " + error.code + " " + error.message); } });

the returned object contains an array of all posts ids containing the search words

[kashwaa]

search has been implemented using key value stores from parse.com, archiver altered to upload search terms to parse.com while creating Json files, excuse the bad design :)

[mtayseer]

I tried to search for my name (including comments) & the results page has 2 entries only.

[kashwaa]

this is due to un-sanitized html code in one of your posts, seems like handlebar doesn't sanitize html automatically, I'll fix this

[kashwaa]

Fixed, will take a look at urlify issue tomorrow