search

justinwalsh / daux.io

Daux.io is an documentation generator that uses a simple folder structure and Markdown files to create custom documentation on the fly. It helps you create great looking documentation in a developer friendly way.
daux.io
4.54k stars 529 forks source link

Plans for search? #18

Closed briannhinton closed 8 years ago

briannhinton commented 11 years ago

Curious if search is in the future roadmap?

justinwalsh commented 11 years ago

Simple way: We could add a custom Google search engine. More info: http://www.google.com/cse/tools/create_onthefly

Long way: Build a custom search parser into the project itself.

I kinda like the idea of the short way... thoughts?

briannhinton commented 11 years ago

Yeah I was thinking of just adding google. However, I was thinking of searching while offline. --  Brian Hinton Sent with Airmail

On July 17, 2013 at 11:20:18 AM, Justin Walsh (notifications@github.com) wrote:

Simple way: We could add a custom Google search engine. More info: http://www.google.com/cse/tools/create_onthefly

Long way: Build a custom search parser into the project itself.

I kinda like the idea of the short way... thoughts?

— Reply to this email directly or view it on GitHub.

justinwalsh commented 11 years ago

I think php search one would be nice for offline, and I don't think it would be that hard to implement a regex search across all of the markdown files. It starts to get tricky when you want to hightlight the results it found, when those are still in markdown.

geebru commented 11 years ago

I don't see highlighting it as important. Usually (unless the document it finds is thousands of words long [in which case perhaps it's time to review how you're breaking things up]) I'm okay with just finding the relevant page. Just basic "bring me to the right page" search functionality would be nice.

justinwalsh commented 11 years ago

I agree with @blackpandemic about the highlighting. I also agree with @briannhinton for the need of offline support. One of the use cases for Daux, is it's ability to run on a private local network for internal documentation. Anyone interested in taking this feature on?

briannhinton commented 11 years ago

I agree. It isn't necessary to highlight the word. Besides once you reach the point where you've found the relevant page, it's possible to do a page specific search with the browser. --  Brian Hinton Sent with Airmail

On July 19, 2013 at 1:11:53 AM, Greg B. (notifications@github.com) wrote:

I don't see highlighting it as important. Usually (unless the document it finds is thousands of words long [in which case perhaps it's time to review how you're breaking things up]) I'm okay with just finding the relevant page. Just basic "bring me to the right page" search functionality would be nice.

— Reply to this email directly or view it on GitHub.

dubcanada commented 11 years ago

I think this is probably one of the most important features. Documentation without a search is just a bunch of text :)

And I personally don't like the idea of "google search" while it may work, it will never work for enterprise or internal stuff.

dkisselev commented 11 years ago

I think the hard part about search will be that the docs are all stored in their own individual files that are rendered on-demand.

Unless we implement some sort of caching, daux would have to open and check every single file, which could get complicated as we add more levels of nav.

dubcanada commented 11 years ago

Hrm... I see your point. Maybe include the ability to harness Apache Solr or Elastic Search?

The only other option is to look at caching the files in a database...

dkisselev commented 10 years ago

For a dirt short-term search, we can just write a quick function to sequentially open every .md file and look for the exact string within it, then generate an HTML page with the Document link + a couple relevant lines.

It'll be slow and resource-intensive though.

pvspencer22 commented 10 years ago

Any update on this feature or examples of how people have used custom google search on their projects?

dkisselev commented 10 years ago

Nobody has actually implemented search as far as I know/have heard. The old points of the issue of data being spread over lots of files still stand though.

Now that someone has developed a script that generates static file, perhaps it's possible to make an add-on to that which indexes the site content to create a search index to make searching happen.

No plans for it at this time though.

nicekiwi commented 10 years ago

This interests me. :p I'm keen to make this happen.

dkisselev commented 10 years ago

I just took a look into this, and it looks like SQLite might be a perfect solution,

http://www.phparch.com/2011/11/full-text-search-with-sqlite/

SQLite should be available with most PHP installations, the only hiccup is that this requires the fts4 extension which might not be available by default.

So for work to do, someone needs to implement the SQLite database that maps page paths from the daux tree object to the sqlite database, then write an extension to the CLI commands that will build the tree, render each page with the markdown parser, and insert the path and full text into a row in the table. Then add a searchbar/page to the theme.

I think for now, the easiest way to do it would be to just delete the search.sqlite database file and create a new one every time the indexing function is run. From what I have seen, Daux is mostly used with less than 100 pages, so it really shouldn't take that long to re-render the search index.

CrockCity commented 10 years ago

Any update on this? This would be huge.

dkisselev commented 10 years ago

I haven't heard of anybody working on it.

My personal docs haven't gotten to the point where they need to be searched yet, though, so I don't think it will be something that I will be working on for a while (ie, months). If anyone else wants to take it on, I'd be happy to help out, but just don't have the time to build it from scratch.

CrockCity commented 10 years ago

Any thoughts on using this? https://github.com/elasticsearch

dkisselev commented 10 years ago

Personally, I think it's a little overkill for daux.

It would require people to install and configure elasticsearch, and have it constantly sitting in memory. And the work required to implement it would be basically identical to the SQLite method. (Getting a CLI script set up that generates the entire tree and every page, and pushes those variables one-by-one as a searchable entry either into ES or SQLite.

Like I said, I don't think daux is used by very many people simultaneously, fr what I've seen, it's 2-10ish developers in small offices that just want something simple to set up. So having ES sitting in memory for little benefit isn't great. It would also make it impossible for people on shared web hosting to use search, since they can't install it. (Whereas sqlite is available by default with many hosts)

On March 19, 2014 8:27:20 AM PDT, CrockCity notifications@github.com wrote:

Any thoughts on using this? https://github.com/elasticsearch

Reply to this email directly or view it on GitHub: https://github.com/justinwalsh/daux.io/issues/18#issuecomment-38064235

CrockCity commented 10 years ago

We have thousands of people searching our daux.io implementation. Love it for the most part, but we really need to add search. I'll let you know if I come up with anything else :-)

Will Crocker | Success Squad | Follow Appboy on Twitterhttps://twitter.com/Appboy https://twitter.com/Appboy Lead Implementation Consultant | @crockerw http://www.twitter.com/crockerw

On Wed, Mar 19, 2014 at 11:36 AM, Denis Kisselev notifications@github.comwrote:

Personally, I think it's a little overkill for daux.

It would require people to install and configure elasticsearch, and have it constantly sitting in memory. And the work required to implement it would be basically identical to the SQLite method. (Getting a CLI script set up that generates the entire tree and every page, and pushes those variables one-by-one as a searchable entry either into ES or SQLite.

Like I said, I don't think daux is used by very many people simultaneously, fr what I've seen, it's 2-10ish developers in small offices that just want something simple to set up. So having ES sitting in memory for little benefit isn't great. It would also make it impossible for people on shared web hosting to use search, since they can't install it. (Whereas sqlite is available by default with many hosts)

On March 19, 2014 8:27:20 AM PDT, CrockCity notifications@github.com wrote:

Any thoughts on using this? https://github.com/elasticsearch

Reply to this email directly or view it on GitHub: https://github.com/justinwalsh/daux.io/issues/18#issuecomment-38064235

Reply to this email directly or view it on GitHubhttps://github.com/justinwalsh/daux.io/issues/18#issuecomment-38065534 .

dkisselev commented 10 years ago

Hmm, thanks for letting me know. I figured Daux wasn't advanced enough to be useful for a large installation like that.

What I suggested earlier was Google custom site search. Especially if your site is public. Downside is that Google only indexes once a month (from what I've heard, anyway). If your Daux site is public and have been caught by Google, adding a search bar to the HTML is easy.

kwilken commented 10 years ago

What about adding http://www.tipue.com/search/docs/

Especially when generating static sites, the search base for this js should be rather easy to generate.

CrockCity commented 10 years ago

I would make sure to make this an option in the config.json. I've actually implemented Google Custom Search in our Daux.io deployment

Will Crocker | Success Squad | Follow Appboy on Twitter https://twitter.com/Appboy https://twitter.com/Appboy Lead Implementation Consultant | @crockerw http://www.twitter.com/crockerw Check out Appboy Academy http://www.appboy.com/academy, our definitive guide to Marketing Automation for Apps.

On Sat, Jun 14, 2014 at 4:55 AM, Kai Wilken notifications@github.com wrote:

What about adding http://www.tipue.com/search/docs/

Especially when generating static sites, the search base for this js should be rather easy to generate.

— Reply to this email directly or view it on GitHub https://github.com/justinwalsh/daux.io/issues/18#issuecomment-46082458.

laxa88 commented 9 years ago

Hi, I implemented a search bar that quick searches the nav-list on the left using jquery. I've made a pull request. I hope it helps some people in need of this simple feature. If the code is inefficient or not practical, I would also appreciate critique/feedback so I can learn. Thank you.

onigoetz commented 8 years ago

Search has been added for generated documentations, live version doesn't have that feature