Getting started with web scraping using Python

[ahmadmardeni1]

Brief Summary:

Web scraping is a technique in which a computer program extracts data from human-readable output coming from a website.

• quetext website which you are using to check our articles for plagiarism is using web scraping.
• flights & hotels booking websites are using it in order to know the lowest price for a ticket, etc.
• a lot of companies are scraping social network profiles in order to determine the posts with the most interactions. then to create strategic marketing decisions.

In this tutorial, I will be using python with beautiful soup library to make the reader understand what is web scraping, how he can use it to gather important information in a good format, and we will create two programs step by step, one of them will be a Covid19 Desktop Notifier which makes a live update on COVID-19 Cases and Deaths for your country using a popup notification on desktop.

Key Takeaways:

• what is a web scraping and how it is working.
• how to check if the website is allowing us to do web scraping, in order to make the operation legal(80-90% of the websites are allowing web scraping)
  • getting started with beautiful soap library and plyer.
  • practicing web scraping using Python in two programs.

References:

N/A

[ninjaginja]

@ahmadmardeni1 We actively sell products that aim to prevent web scraping. I see that one of your takeaways talks about ensuring permissions around web scraping, but I just want to be sure that this is front and center with the article. Is there a way to reframe the article as 'Building a COVID-19 Desktop Notifier with Python', rather than framing it within the context of web scraping?

[ahmadmardeni1]

@ninjaginja my aim was to focus on legal issues in order to prevent new learners from doing web scraping on random websites, so I wrote about 2 methods in order to know if the website is allowing us or not. Also, I can change the tutorial just to "Build a COVID-19 Desktop Notifier". let me know if you gave me permission to write about web scraping or not, I think I can give the reader a good value because I'm an expert with it.

[ninjaginja]

Yes, the topic is okay as long as you lead with the legal issues.