search

ArthurHeitmann / arctic_shift

Making Reddit data accessible to researchers, moderators and everyone else. Interact with the data through large dumps, an API or web interface.
https://arctic-shift.photon-reddit.com
285 stars 21 forks source link

Daily data archiving #26

Open xuyang-stack opened 5 days ago

xuyang-stack commented 5 days ago

Can you discuss how to obtain daily archived data, because monthly archives sometimes have poor real-time performance. At present, it seems that most APIs on the Internet can only search data given search conditions.

ArthurHeitmann commented 5 days ago

You can use the /search APIs before or after parameters to iterate over all data. However with over 1 mil new posts and 10 mil new comments a day, you will have to be patient.