search

eloquence / freeyourstuff.cc

freeyourstuff.cc - universal content liberation
Creative Commons Zero v1.0 Universal
79 stars 4 forks source link

Download Limit Reached #127

Open gordongmilleriii opened 4 years ago

gordongmilleriii commented 4 years ago

I have 10,653 answers on Quora and the most I can download is 1,297. Not sure what the issue is. Also, you have 100 items per page, but with 10,000 answers, that would be 100+ pages. How about an option for ALL pages, please.

https://www.quora.com/profile/Gordon-Miller-21/answers

eloquence commented 4 years ago

Hi Gordon,

Sorry you ran into trouble downloading your answer set. I was able to reproduce the issue. Quora itself starts to return more and more gateway timeout errors the further down the answer list we go.

This is a new issue (the largest answer-set I know of that's been successfully downloaded has more than 10,000 answers in it). It may be temporary, and could indicate problems with Quora's servers that will be resolved in a few days.

I was able to download 3,800 answers by massively increasing the retry limit. I'll let it run overnight without a limit to see if I can download them all for you, so I can at least send you a copy. BTW, your profile page indicates that you have 7,997 answers. Is there some stuff I'm not able to see?

How about an option for ALL pages, please.

Not sure what you mean here -- this is a third party project and not affiliated with Quora. Because Quora does not provide an API, we can only work with whatever limits Quora puts in place for loading / displaying answers through the browser. If there's a view that lets you paginate through answers more effectively than the profile page, please don't hesitate to let me know -- I'd love to find a way to do this more efficiently.