Closed mohammad-fahs closed 1 month ago
Website: shadi najjar website A website for selling electronics, i scraped the Ipad Category
Scraping Application class: package com.nader.web_scraper;
import org.springframework.boot.CommandLineRunner; import org.springframework.stereotype.Component;
import java.util.List; import java.util.Scanner;
@Component public class ScrapingApplication implements CommandLineRunner { ScrapingService scrapingService = new ScrapingService();
@Override
public void run(String... args) throws Exception {
Scanner scanner = new Scanner(System.in);
while (true) {
System.out.println("enter link to scrape its data or exit to stop");
String link = scanner.nextLine();
if (link.equals("exit"))
break;
List<Phone> phoneList = scrapingService.scrapePhones(link);
for (Phone phone : phoneList) {
System.out.println(phone.toString());
}
}
}
}
Task 2 :
Objective:
In this task, you will initialize a new Spring Boot project, add the JSoup dependency, and write a simple Java program to scrape data from a website of your choice. The goal is to get hands-on experience with setting up a Spring Boot project, using an external library (JSoup), and applying web scraping techniques.
Instructions:
com.yourname
web-scraper
Web Scraper
com.yourname.webscraper
Spring Web
dependency (to allow adding more features later).pom.xml
file in the root directory of your project.<dependencies>
tag:pom.xml
file and allow IntelliJ to update the Maven project to download the JSoup library.com.yourname.webscraper
package that implements theCommandLineRunner
interface.run
method, use JSoup to connect to the website you chose and scrape the data.Submit Your Work:
CommandLineRunner
.Resources that can help: