sopel-irc / sopel

:robot::speech_balloon: An easy-to-use and highly extensible IRC Bot framework. Formerly Willie.
https://sopel.chat
Other
949 stars 403 forks source link

search, xkcd: built-in search plugin has external replacement #2613

Open dgw opened 1 month ago

dgw commented 1 month ago

Description

The built-in search has been unreliable, often yielding no results because it probably gets blocked from scraping Bing or DuckDuckGo SERPs. (This mechanism isn't entirely clear, as it still does work from some machines—including places like Gitpod cloud development servers that presumably live in a higher-profile public cloud than any of the bots with problems.)

However, a near-scratch rewrite at sopel-irc/sopel-search using a library to access DDG seems to be significantly more reliable, and lets us drop xmltodict as a core dependency; search.py was the last place using it.

This patch also removes sopel.builtins.search.duck_search() as a fallback from the xkcd plugin, for better or worse putting all of its proverbial eggs in the searchxkcd.com basket. (I reworked the error-handling there a little, too.)

Checklist

Notes

I see no problem with doing this in 8.1, but am open (as always) to other opinions 😸

dgw commented 1 month ago

Staying in draft until type-checking errors are fixed, which should be accomplished by #2614.

dgw commented 1 month ago

Rebased onto master after merging #2614. Finally ready to review 😁