Open adeak opened 2 years ago
As of this date, I had to downgrade pillow to pillow==9.5.0
. I additionally had to fix numpy
so that it works with MacOS M1/M2 chips: numpy==1.24.4
and I had to additionally install lxml
: lxml==5.2.2
. Please consider making a change to your PR and modify the requirements.txt
file accordingly.
The profile pages on SO/SE have been completely rewritten (see announcement from December 7, 2021), which means much of this library has to be rewritten.
Since the profile pages are an opaque mess of nested divs now (starting to look a lot like twitter HTML), the easiest approach I could find was to find divs with titles like this:
One tag on the tag page gets one of these divs, and this already gives us the tag score. Inside there's a tag with the tag's name for text. I didn't want to rely on those random-looking strings in the
class
attribute.I've also changed a handful of things (some of them stylistic):
return
s.No doubt the company will add arbitrary small changes in a few weeks just to break scrapers like this. Until then this should work (even if slow due to the throttling/pushbacks).