shadowmoose / RedditDownloader

Scrapes Reddit to download media of your choice.
1.11k stars 99 forks source link

AttributeError: 'NoneType' object has no attribute 'items' #181

Open callanrocks opened 3 years ago

callanrocks commented 3 years ago

Describe the bug

Was using it down download an entire subreddits shitposting for important archival purposes, randomly stops and throws the below error when it comes across a post it doesn't like but can't narrow down which

Environment Info

Screenshots/Information

Process RedditElementLoader: Traceback (most recent call last): File "filepath\Python\lib\multiprocessing\process.py", line 258, in _bootstrap self.run() File filepath\processing\redditloader.py", line 30, in run self.load() File filepath\processing\redditloader.py", line 51, in load self._scan_sources() File filepath\processing\redditloader.py", line 65, in _scan_sources for r in source.get_elements(): File filepath\sources\user_posts_source.py", line 14, in get_elements find_comments=self.data['scan_comments']): File filepath\static\praw_wrapper.py", line 148, in user_posts yield RedditElement(c) File filepath\processing\wrappers\redditelement.py", line 39, in init self.detect_type(obj) File filepath\processing\wrappers\redditelement.py", line 56, in detect_type self._submission(obj) File filepath\processing\wrappers\redditelement.py", line 133, in _submission for k, img in post.media_metadata.items(): AttributeError: 'NoneType' object has no attribute 'items'

MaeInHeels commented 3 years ago

Any updates on this?

I'm still having the same issue on 3.1.4:

(Warning, NSFW ahead)

YTDL: ERROR: Unable to download JSON metadata: HTTP Error 404: Not Found (caused by <HTTPError 404: 'Not Found'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output. [https://gfycat.com/DeafeningHugeFrigatebird]
Process RedditElementLoader:
Traceback (most recent call last):
  File "multiprocessing\process.py", line 297, in _bootstrap
  File "processing\redditloader.py", line 30, in run
  File "processing\redditloader.py", line 51, in load
  File "processing\redditloader.py", line 65, in _scan_sources
  File "sources\user_posts_source.py", line 14, in get_elements
  File "static\praw_wrapper.py", line 149, in user_posts
  File "processing\wrappers\redditelement.py", line 39, in __init__
  File "processing\wrappers\redditelement.py", line 56, in detect_type
  File "processing\wrappers\redditelement.py", line 133, in _submission
AttributeError: 'NoneType' object has no attribute 'items'

The issue happens when downloading user posts from user adamant_cupcake, here's the configuration:

{
    "alias": "Adamant_Cupcake",
    "data": {
        "scan_comments": false,
        "scan_submissions": true,
        "user": "Adamant_Cupcake"
    },
    "filters": {},
    "type": "user-posts-source"
}
shadowmoose commented 3 years ago

Thanks for the config. I'll take a look - this was something I thought I'd fixed in the last update, but with test data now I should be able to dig deeper.