search

mattpodolak / pmaw

A multithread Pushshift.io API Wrapper for reddit.com comment and submission searches.
MIT License
213 stars 28 forks source link

Subreddit restriction is not an exact name match and includes subreddits with a superset of that name #60

Open brendon-wong opened 1 year ago

brendon-wong commented 1 year ago

Hi all! I was trying to restrict to a specific subreddit "web10" and I noticed that content from other subreddits with a name that includes the text "web10" is also coming up! I assume this is not the intended behavior because it could cause a lot of unrelated results to appear, especially if the subreddit name has a common word like "science" which is used by many subreddits besides r/science.

In this example (https://api.pushshift.io/reddit/search/submission?subreddit=web10), content from r/web10, r/u_Psychological-Web10, and r/u_ronaldo-web10 appears. r/u_Psychological-Web10 is a subreddit, and r/u_ronaldo-web10 is handled differently by Reddit (Reddit displays a page indicating the user has been banned, rather than a page indicating the subreddit doesn't exist) so perhaps it was previously a subreddit.

eddvrs commented 1 year ago

This appears to be an issue with the API itself rather than PMAW. PMAW can only reflect what's returned by Pushshift. There's a similar issue with searches by author.

brendon-wong commented 1 year ago

My bad, I even used a link to the API itself as an example of the issue! This issue first arose when using pmaw, so I must've opened the issue here first. Just created an issue in the Pushshift repo itself.

If Pushshift doesn't fix this issue, I wonder if it'd make since for pmaw to implement filtering to get an exact word match?

brendon-wong commented 1 year ago

Here is the issue in the Pushift repo: https://github.com/pushshift/api/issues/144