search

uxdxdev / orca

Download your reddit data.
18 stars 1 forks source link

No comment data before ~2018 #1

Closed taylorthurlow closed 3 years ago

taylorthurlow commented 3 years ago

I don't know if it's limited to a certain date or a certain number of comments, but the earliest comment I have is from October of 2018, and my account was created in 2011.

uxdxdev commented 3 years ago

I don't know if it's limited to a certain date or a certain number of comments, but the earliest comment I have is from October of 2018, and my account was created in 2011.

Thanks for reporting this, can you tell me how many comments were recorded? I've seen reports that the Reddit API limits requests to 1000.

taylorthurlow commented 3 years ago

if grepping the output for Comment: is an accurate count, then I got 931.

uxdxdev commented 3 years ago

if grepping the output for Comment: is an accurate count, then I got 931.

Did you only fetch comments? e.g. using --data=comments or did you fetch all data types? e.g. comments, upvotes, saves, and submissions?

Orca will fetch all data types by default.

Maybe the remaining 69 requests were made to retrieve the other data types?

taylorthurlow commented 3 years ago

I fetched comments and submissions, and I got 309 submissions dating back to when my account was created.

uxdxdev commented 3 years ago

I fetched comments and submissions, and I got 309 submissions dating back to when my account was created.

I'm wondering now if maybe the Reddit API is limiting a single data type to 1000, and where you got 931 comments returned it also counts comments that were deleted. Do you think that you might have 69+ deleted comments?

Try using https://www.reddit.com/settings/data-request to review your data, in my data it shows comments and has entries for deleted comments too.

uxdxdev commented 3 years ago

Because Orca is using https://github.com/not-an-aardvark/snoowrap under the hood, I think this bug should be reported on the snoowrap repo instead.

I am interested to know if there is a workaround.