Add duckduckgo as an upstream search engine for text

[305a385]

Add duckduckgo as upstream search engine

Info:

• the issue I made for this got closed because according to @hnhx

  DDG has horrible text results so thats why i didnt bother with it.

  However, because I'm making a pull request, I'm bothering with it.

• This is not made for librex to rank google and duckduckgo but rather for the instance hoster to choose. These means that if the instance hoster is getting ratelimited by google they can use duckduckgo instead
• This scrapes duckduckgo rather than use the API, because the API sort of sucks

Tasks

• [ ] Update the wiki to reflect changes
• [ ] Allow other search engines to be used as a fallback
• [x] Add support for querying multiple pages
• [ ] Add safe search support EDIT: It seems like duckduckgo html doesnt support safe search
• [x] make search results not use duckduckgo for a link proxy (e.g. links will currently redirect to "http://duckduckgo.com/l/?uddg=https%3A%2F%2Fen.wikipedia.org" instead of "https://en.wikipedia.org")
• [x] Add option in config.php to switch between search engines
• [x] actually add duckduckgo as a search engine

Testing

(this assumes your config.php has just been copied from config.php.example) search for the line "text" => "google" and change it to "text" => "duckduckgo"

Solves: #232

[hnhx]

Why did you make the preferred engine an array if its only 1 value lol Also this doesnt really make ddg a fallback nor you can get both search results at once, you have to manually change the config file if google blocks you.

[305a385]

Why did you make the preferred engine an array if its only 1 value lol

In case we decide to add something like this for images, etc. It makes it easily upgradable

this doesnt really make ddg a fallback nor you can get both search results at once, you have to manually change the config file if google blocks you.

I thought of that but forgot to add it to the tasks list

[codedipper]

This is a great feature, thank you! Another thing for public instance owners, there should be a section in the README that makes it clear what the default engine is.

Also using DuckDuckGo on my onion instance because I do searches on onion and I want my search engine to actually be usable.

[ghjardim]

@codedipper I agree. Using DuckDuckGo enables webscraping through proxies easier.

[ghjardim]

@305a385, it seems to have a problem with the duckduckgo code. I've tried to query "linux-libre", and only the Linux-Libre card was displayed, while the actual results were omitted. Doing the same query in public instances (that use Google engine) doesn't lead to this problem, and the results are shown.

[305a385]

Sorry to everyone for the late reply

@hnhx was right, duckduckgo gives terrible results, more specifically html.duckduckgo.com gives terrible results. It doesn't even support safe search!

If anyone wants to continue this work I would recommend restarting this using something like brave search instead

[codedipper]

Hello, we implemented your change into Ahwxorg/LibreY#2! LibreY is a fork of LibreX to remove unwanted changes and because the original LibreX repository (hnhx/librex) appears to be dead. We love you <3