search

jhvst / vertigo

Blog engine in Go (golang)
https://toldjuuso.github.io/vertigo
MIT License
263 stars 24 forks source link

Search and clauses #40

Closed jhvst closed 8 years ago

jhvst commented 9 years ago

Currently the search does not work if you search with clauses instead of single word.

This is unexpected behavior.

jayrox commented 9 years ago

example?

jhvst commented 9 years ago

Search "Blocking does not work anymore in the recent versions of" on my blog

jayrox commented 9 years ago

hmm are you sure its not something else causing than a clause.

examples: "Blocking" - returns 3 results "Blocking does" - returns 1 "Blocking html" - returns 1 "Blocking html links" - returns 0

seems i cant do any searches with 3 or more words in the search field. is something possibly trimming the search query?

jhvst commented 9 years ago

It's lack of features which is causing this.

In posts.go where the Get method is declared the contents of each published post is iterated word by word to achieve fuzzy search. This is beneficial if you are only using a single keyword to find content. For example, if you search for 'twatch' instead of 'twitch' on my blog, you will still get the Twitch post as a result, even though the post does not actually contain the keyword (this is also the reason why I do not plan to add tags or hashtag options to posts).

But when you give it more words than one, the fuzzy search algorithm does not work, since it is meant to compare single words. Searching for "Blocking does not work anymore in the recent versions of" will eventually compare "Blocking" with "Blocking does not work anymore in the recent versions of", which breaks the algorithm since the second phrase is so long.

This could be fixed by first searching the phrase as it is given by the user, after which it will run fuzzy search if nothing is found.

However, I'd prefer not to mimic Google too much, since that's what the user will ultimately use anyway if they do not find the content they are looking for.

Whether it is common for users to search content from blogs with specific keywords happens to be something I do not have much data of, but I'd guess it would be rather rare.

I'd rather hold on to the search functionality since it is one feature a dynamic blog can offer over a static one, without resorting to embedded Google search.

jayrox commented 9 years ago

what are the cons to just running both search methods? if fuzzy returns zero results, run full search. the full search could even be handled by the sql engine which would likely be faster than loading all of the entries into the go app and then iterating the subjects and bodies for the string.

something like 

 SELECT * FROM posts WHERE subject LIKE "%query%" OR body LIKE "%query%"

thats not exactly right but leads to the right idea i believe.

jhvst commented 9 years ago

I'll look into how I could leverage the DB on this. Your suggestion sounds good to me.

jhvst commented 9 years ago

I've been developing the underlying algorithm package. The changes to the actual search routes are still undone, as the conversion to idiomatic HTTP stack seems to require a lot more attention than I initially thought.

jayrox commented 9 years ago

does the code in the alice repo help at all with the idiomatic routes?

jhvst commented 9 years ago

It will let me just copy-and-paste code from the repo, which will speed things up a lot. Thank you for that.

However, before that I'd like to finalize #37. I got tests to pass on dev branch yesterday, which means I can start working on #37 next. After that I'll merge the alice branch and handle this issue. After those things I can start looking at a new public release.

jayrox commented 9 years ago

i can help with #37, i'll look at it a bit today if a get a chance

On Wed, Jul 8, 2015 at 12:54 AM, Juuso Haavisto notifications@github.com wrote:

It will let me just copy-and-paste code from the repo, which will speed things up a lot. Thank you for that.

However, before that I'd like to finalize #37 https://github.com/9uuso/vertigo/issues/37. I got tests to pass on dev branch yesterday, which means I can start working on #37 https://github.com/9uuso/vertigo/issues/37 next. After that I'll merge the alice branch and handle this issue. After those things I can start looking at a new public release.

— Reply to this email directly or view it on GitHub https://github.com/9uuso/vertigo/issues/40#issuecomment-119433136.

jhvst commented 9 years ago

Cool, I'd appreciate that. :+1:

jhvst commented 9 years ago

@jayrox Have you had the time to start the merge? I'll start it in a new branch unless you already have.

jayrox commented 9 years ago

i placed my files in the timezone branch, if you to look it over before merge. last time i checked it looked like it could be merged without issue

jhvst commented 9 years ago

Oh, I somehow mixed the issues.

I'll start looking at #27 next. I'll use the alice repo as the base for that. Expect new branch in a few days.

jayrox commented 9 years ago

sounds good, the code regarding the timezone was based on the martini version but it shouldnt take much to move it over. i'm currently in another state for work until thursday. if i get some time i'll try to look at moving it to the alice branch. if your fork up to date? i'll clone it and get to work on that.

On Mon, Jul 13, 2015 at 12:55 PM, Juuso Haavisto notifications@github.com wrote:

Oh, I somehow mixed the issues.

I'll start looking at #27 https://github.com/9uuso/vertigo/issues/27 next. I'll use the alice repo as the base for that. Expect new branch in a few days.

— Reply to this email directly or view it on GitHub https://github.com/9uuso/vertigo/issues/40#issuecomment-120992442.

jhvst commented 9 years ago

Though I'm going to try develop my own rendering library if that's what's needed to get .LoggedIn helpers etc to work well.

jayrox commented 9 years ago

Check the Alice repo, the logging stuff works.. I have it commented out cause I was trying to get the tests to pass On Jul 13, 2015 4:00 PM, "Juuso Haavisto" notifications@github.com wrote:

Though I'm going to try develop my own rendering library if that's what's needed to get .LoggedIn helpers etc to work well.

— Reply to this email directly or view it on GitHub https://github.com/9uuso/vertigo/issues/40#issuecomment-121038192.

jhvst commented 9 years ago

probably wisest to use this: https://github.com/blevesearch/bleve