The built-in search has been unreliable, often yielding no results because it probably gets blocked from scraping Bing or DuckDuckGo SERPs. (This mechanism isn't entirely clear, as it still does work from some machines—including places like Gitpod cloud development servers that presumably live in a higher-profile public cloud than any of the bots with problems.)
However, a near-scratch rewrite at sopel-irc/sopel-search using a library to access DDG seems to be significantly more reliable, and lets us drop xmltodict as a core dependency; search.py was the last place using it.
This patch also removes sopel.builtins.search.duck_search() as a fallback from the xkcd plugin, for better or worse putting all of its proverbial eggs in the searchxkcd.com basket. (I reworked the error-handling there a little, too.)
Description
The built-in
search
has been unreliable, often yielding no results because it probably gets blocked from scraping Bing or DuckDuckGo SERPs. (This mechanism isn't entirely clear, as it still does work from some machines—including places like Gitpod cloud development servers that presumably live in a higher-profile public cloud than any of the bots with problems.)However, a near-scratch rewrite at sopel-irc/sopel-search using a library to access DDG seems to be significantly more reliable, and lets us drop
xmltodict
as a core dependency;search.py
was the last place using it.This patch also removes
sopel.builtins.search.duck_search()
as a fallback from thexkcd
plugin, for better or worse putting all of its proverbial eggs in thesearchxkcd.com
basket. (I reworked the error-handling there a little, too.)Checklist
make qa
(runsmake lint
andmake test
)Notes
I see no problem with doing this in 8.1, but am open (as always) to other opinions 😸