Discuss the regex used to filter posts/comments

[sabjoslo]

A couple things I noticed/questions about the regex:

• It doesn't match the string r"[same-sex|same sex] marriage".
• Should we include patterns like "LGBTQ rights"?

[BabakHemmatian]

Thanks for the helpful comments! I was so focused on getting the code to work that I neglected the more theoretical aspects of the work.

• It should certainly include same-sex and same sex. Thanks for noticing that. I'll add them to the code.
• I think LGBTQ rights is too broad. For instance, much of the discussion in more recent times might revolve around trans rights, which is not too related to gay marriage. We could add combinations of LGBT.*? and marri|marry to the regex. If both are in a comment, chances are it is related to gay marriage.

[sabjoslo]

Sounds good! We might also want to try to match (not necessarily as stand-alone strings): "civil union", "lesbian", and probably many other strings I'm not thinking of.

[sabjoslo]

A couple more comments on this:

• I've added a function getFilterRegex() to the file utils.py that returns the same value that's held by the variable GAYMAR in parsereddit. In other words, from utils import *; getFilterRegex()==GAYMAR. I've written code that imports the regex using this function. So if you both set GAYMAR=getFilterRegex() and make changes to the regex within the body of getFilterRegex(), the changes will be visible to both of us.
• c330441 does what I've described in my most recent comment on #2. I'm going to wait to run it until we've got the regex sorted out.
• If we're filtering posts from major news sources, I don't think we need to include slurs in the regex.