search

radiolarian / AO3Scraper

A Python scraper for getting fan fiction content and metadata from Archive of Our Own.
175 stars 56 forks source link

Feature: Scrape users who have authored, kudo-ed, bookmarked #9

Closed bianchi-dy closed 3 years ago

bianchi-dy commented 4 years ago

This is a ... New feature (non-breaking change which adds functionality)

What does this PR do? Scrapes the following information:

How was the change made? Added new functions and columns per item, i.e. get_authors, get_kudos, get_bookmarks. get_bookmarks is nested as a sort of hack since it needs to go through each page to extract the usernames.

Things I'd like feedback on With second-order features such as bookmarks, kudos, etc., how should their columns be named?

What next? Maybe extract comments from bookmarks or comments on work. But the threads might be a pain to store, so I welcome discussion on what's the best data structure for them.

bianchi-dy commented 4 years ago

@radiolarian @ssterman just a quick ping in case you haven't seen this yet! might be useful for others. :)

cspenn commented 4 years ago

I'm using your PR, and it's FANTASTIC!

radiolarian commented 3 years ago

Sorry for the super late accept. Thank you so much, this seems like an awesome feature! I'll also update the readme.