search

openzim / ifixit

iFixit to ZIM scraper
GNU General Public License v3.0
24 stars 3 forks source link

Scrape users #35

Closed benoit74 closed 2 years ago

benoit74 commented 2 years ago

Fix #13

benoit74 commented 2 years ago

Still missing the addition of user links in existing template + the rewriting of user URLs (which unfortunately do not follow the previous convention that the interesting part of the URL is placed last. (cf. /User/About/1568537/Annika+Faelker vs /Tutoriel/Google+Pixel+4+Battery+Replacement/136194) ...

benoit74 commented 2 years ago

Would probably require the use of a SQLite DB to store info in order to retrieve it elsewhere + this means retrieving every content before rendering it ... For instance we need all user info to render a guide author (we need its badges, ...)

benoit74 commented 2 years ago

We won't retrieve the user list in the end, there is way too many "lurking" or spam users. We will build the list of users only a unexpected ones, based on the fact that we built a link for them

benoit74 commented 2 years ago

First tests ok, full run under way ...