praw-dev / praw

PRAW, an acronym for "Python Reddit API Wrapper", is a python package that allows for simple access to Reddit's API.
http://praw.readthedocs.io/
BSD 2-Clause "Simplified" License
3.4k stars 453 forks source link

`DuplicateReplaceException` being raised unexpectedly #1942

Closed cutonbuminband closed 1 year ago

cutonbuminband commented 1 year ago

Describe the Bug

I've encountered something strange, and I'm not even sure whether it's a reddit error or a praw error.

I maintain a set of tools to navigate and keep track of the deeply nested threads at r/counting. As part of that I have a script running that automatically updates a directory of the threads, and yesterday it started failing with an error I've never seen before, namely a DuplicateReplaceException. I have recently updated the code, so I rolled back to a previous version, expecting the error to go away, but it didn't.

Digging in deeper, I was able to reproduce the error without referencing any of my existing code. In a fresh virtual environment with praw 7.6.1 installed this code:

import praw

reddit = praw.Reddit(
    client_id=XXXX,
    user_agent=XXXX,
    refresh_token=XXXX,
    client_secret=None,
)

comment = reddit.comment("j9euqfn")
comment.refresh()
replies = comment.replies
replies.replace_more(limit=None)

fails. Looking at replies.list() before I call replace more I see two comments (ids j9ev71v and j9evimq) and then a MoreComments object. If I add a print statement at praw.models.comment_forest:195 to see what comment is the immediate cause of the failure, I see comment id j9evimq, which is indeed already present in the replies.

Somewhere it seems that reddit is sending something that praw is not expecting.

Has anyone else experienced something similar? Is this a known issue with a known workaround?

Desired Result

No exception should be raised

Code to reproduce the bug

import praw

reddit = praw.Reddit(
    client_id=XXXX,
    user_agent=XXXX,
    refresh_token=XXXX,
    client_secret=None,
)

comment = reddit.comment("j9euqfn")
comment.refresh()
replies = comment.replies
replies.replace_more(limit=None)

The Reddit() initialization in my code example does not include the following parameters to prevent credential leakage:

client_secret, password, or refresh_token.

Relevant Logs

Traceback (most recent call last):
  File "debug.py", line 15, in <module>
    replies.replace_more(limit=None)
  File ".venv/lib/python3.10/site-packages/praw/util/deprecate_args.py", line 43, in wrapped
    return func(**dict(zip(_old_args, args)), **kwargs)
  File ".venv/lib/python3.10/site-packages/praw/models/comment_forest.py", line 196, in replace_more
    self._insert_comment(comment)
  File ".venv/lib/python3.10/site-packages/praw/models/comment_forest.py", line 80, in _insert_comment
    raise DuplicateReplaceException
praw.exceptions.DuplicateReplaceException: A duplicate comment has been detected. Are you attempting to call 'replace_more_comments' more than once?

This code has previously worked as intended.

Yes

Operating System/Environment

Ubuntu 22.04.2

Python Version

Python 3.10.6

PRAW Version

7.6.1

Prawcore Version

2.3.0

Anything else?

No response

cutonbuminband commented 1 year ago

I still see the problem on comments that were made after the outage on the 21st. For example, comment j9svsj0, which was made earlier today.

I don't know the praw internals, so I'm not sure what the MoreComments object is supposed to look like before expansion. If I try to display it, I see <MoreComments count=0, children=[]>. Is that normal?

cutonbuminband commented 1 year ago

As an update, the problem is still present in a fresh install of praw 7.7.0

github-actions[bot] commented 1 year ago

This issue is stale because it has been open for 30 days with no activity.

Remove the Stale label or comment or this will be closed in 30 days.

cutonbuminband commented 1 year ago

The problem is still occurring. It seems that for deep comment chains, reddit has started sometimes putting the "continue this thread" link at the bottom of the page (like this) instead of attached to the parent comment. This breaks the assumptions in replace_more

github-actions[bot] commented 1 year ago

This issue is stale because it has been open for 30 days with no activity.

Remove the Stale label or comment or this will be closed in 30 days.

github-actions[bot] commented 1 year ago

This issue was closed because it has been stale for 30 days with no activity.