search

johncoleman83 / domain_scraper

Scrapes domains for broken links, emails & social media links (uses beautifulsoup)
MIT License
2 stars 3 forks source link

1: Added entry point for all scripts #2

Closed edvein-rin closed 5 years ago

edvein-rin commented 5 years ago

It works, but... I didn't add command for "scrape_url_and_check_broken_links" because you should separate them. The same with "scrape_email_and_socmedia". It really would be better if modules were like these: "scrape email", "scrape socmedia". All funcs should do only one thing, the same with scripts. Just add a chain querries like: "scrape url" + "scrape socmedia" + "check broken links". And don't add "NO new links". Add "WITH new links". It's logical, and easier to add in the chain: "scrape url" + "new links". Sorry, for not writing all commands, but it won't be needable if you use separated modules. Good luck! Issue #1

johncoleman83 commented 5 years ago

Thanks sorry to delay, looks good. I will review this soon and likely merge. I may have a few suggestions, but will probably just merge and fix myself. Looks really good though, just got to go now but will review soon.

johncoleman83 commented 5 years ago

Thanks for your contribution! Even though this is a total WIP and probably should not yet be merged, I see what you are going for and will quickly revise the things lacking in these updates.

edvein-rin commented 5 years ago

Thank you for your feedback, and sorry for any troubles. I glad to help Open Source.

johncoleman83 commented 5 years ago

You're welcome. Also, thanks for your support. Feel free to add yourself as a contributor.