dmarx / psaw

Python Pushshift.io API Wrapper (for comment/submission search)
BSD 2-Clause "Simplified" License
361 stars 53 forks source link

AttributeError: 'submission' object has no attribute 'selftext' #95

Closed jaevibing closed 3 years ago

jaevibing commented 3 years ago

Hello, I am running a script which gets all posts from a subreddit and puts them in a csv. I got this error. How can I fix this?


AttributeError                            Traceback (most recent call last)
<ipython-input-4-aa75b322a6bb> in <module>()
     11     writerobj.writerow(["subreddit","body","time","id","url","author"])
     12     for submission in gen:
---> 13         writerobj.writerow([submission.subreddit, submission.selftext, submission.created_utc, submission.id, submission.url, submission.author])

AttributeError: 'submission' object has no attribute 'selftext' ```
jaevibing commented 3 years ago

It works fine with other posts but for some reason it doesn't work on this post.

markanewman commented 3 years ago

I also encountered this issue when writing JSON. It seems that some submissions in pushshift.io will be missing the JSON element selftext when the post has no body. My problem is a deleted post causing this. The real rub is when it happens to be the first element on a boundary. When that happens, all the other elements will be missing that element too, even if a manual call to the pushshift.io API would return them.

calling api with following arguments:
   {'after': 1609477200,
    'before': 1612155599,
    'filter': ['id', 'created_utc', 'author', 'title', 'selftext'],
    'limit': 1000000,
    'subreddit': ['COVID19_support']}
server did not return following fields: ['selftext']
jaevibing commented 3 years ago

Fixed I added

try:
    //code
except:
   pass

to my code and it works fine.