[RFC] Don't blag Medium comments

[kamalmarhubi]

DO NOT MERGE since there's nothing here.

This is an empty PR for now, because I want to get feedback on what I'm planning to do since it's a bit hackish. But I find this Medium nonsense really detracts from joy of perusing the blogging stream, so maybe a hack is warranted.

The main observation: Medium "comment" posts have a link back to the post that is being replied to as part of their header, while Medium "post" posts do not. So a comment's header looks like this:

header
├── div
│   └── div
│       └── div
│           └── ...
└── a
    └── div
        └── div
            └── ...

while an ordinary post's header looks like this:

header
└── div
    └── div
        └── div
            └── ...

My proposed solution:

• add a non-null BooleanField called ignored with default False to the Post model
• modify crawlposts to treat links specially:
  • fetch the link
  • parse the HTML
  • query for CSS header > a (direct descendent) or XPATH //header/a (whichever is easier given choice of HTML parsing library)
  • if there is a result, set ignored to True in the generated Post
  • only send to Zulip if the post is not ignored

fixes #143

---

• [ ] Test
• [ ] Screenshot
• [ ] AUTHORS.md entry
• [ ] CHANGES.md entry
• [ ] Documentation

[punchagan]

This is an empty PR for now, because I want to get feedback on what I'm planning to do since it's a bit hackish. But I find this Medium nonsense really detracts from joy of perusing the blogging stream, so maybe a hack is warranted.

I agree that a hack is probably warranted. User experience over beautiful code.

@kamalmarhubi Would you have the time to implement this? Or would you like me to?

[kamalmarhubi]

I can do it or not, up to you!

[punchagan]

I can do it or not, up to you!

@kamalmarhubi I will definitely not be able to get to this for another couple of weeks. Would you be willing to work on this? Sorry for the super delayed response.

[punchagan]

@kamalmarhubi I started using Zulip after a long break, and found the medium comments annoying. So, I finally got around to fixing this bug. The fix is very similar to what you suggest, but it looks at the json that the Medium pages contain, rather than the html structure.

[kamalmarhubi]

Nice!