search

jtsay362 / solveforall

solveforall Issue Tracker
1 stars 0 forks source link

DuckDuckGo Answer Generator not working #4

Open SandNerd opened 7 years ago

SandNerd commented 7 years ago

Repro Steps

  1. Create an engine soley for DuckDuckGo as in here
  2. Either:
    • Within the engine page use Try it out with any query (e.g. ocr)
    • Further set Active by default? and Use for reference? both to true, save engine and use SolveForAll upper search box with any query

Expected: Top results from DuckDuckGo to be shown on search page Actual: No results from DuckDuckGo. Only mentions Try your search on: DuckDuckGo at the top

Enviornment

jtsay362 commented 7 years ago

Hi,

Unfortunately, you have experienced the expected behavior. There are two issues that prevent you from getting what you expect:

1) Normally, URI template answer generators such as the DuckDuckGo one only produce links to the search page. However, it is possible to have Solve for All fetch the search page and extract the relevance links by enabling "Deep Search". 2) Unfortunately, DuckDuckGo doesn't all scraping of its results and as a search engine that also respects user privacy, I feel it would be unethical to do so. Other sites (mostly e-commerce sites) are scrapable though. Try this URL if you are curious:

https://solveforall.com/answers.do?q=logitech+mouse&type=answers&engines=107

and enable "Deep Search". Then reload (sorry). I'm still fleshing out this feature, but this should give you a preview. Thanks for trying out Solve for All!

SandNerd commented 7 years ago

I'd say then that all non-scrapable sources (for whatever reason) should be their own category of sort to notify the user.

jtsay362 commented 7 years ago

Good suggestion. Many of the non-scrapable sources are in the "Search" category, but that includes scrapable ones as well. In the future, I'll be added a label that indicates that a source is scrapable or not and I think this should be satisfactory to you. When that is done, I will close this issue.