search

NISH1001 / medium-crawler

A crawler for scraping posts from medium.com
GNU General Public License v3.0
63 stars 15 forks source link

[Crawler Issue] Now the crawler doesn't fetch anything #4

Open NISH1001 opened 5 years ago

NISH1001 commented 5 years ago

As of writing this, it seems that there have been changes in div and other HTML elements in "medium dot com". This renders the crawler worthless for now. I will fix this when I have some leisure time to invest in this project.

:)

VladislavSmolyanoy commented 1 year ago

Any updates? :(

NISH1001 commented 1 year ago

@VladislavSmolyanoy Hi. I apologize but I have stopped working on this project. Maybe you can fork and try changing the crawler? I am sure there are better alternatives than this. Something like scrapy might do? :)

VladislavSmolyanoy commented 1 year ago

I was just trying to save some time and find a library that does that... I guess I'll have to build something myself then 😊

NISH1001 commented 1 year ago

I think you can start with scrapy. The only caveat is how you can bypass the paywall on medium. Medium today has become a mess of UI elements.