search

Recode-Hive / Scrape-ML

For new data generation Semi-supervised-sequence-learning-Project we have writtern a python script to fetch📊, data from the 💻, imdb website 🌐 and converted into txt files.
https://scrape-ml.streamlit.app/
MIT License
80 stars 117 forks source link

Adding a Updated_Scrapper file #161

Closed Harshitmishra001 closed 3 weeks ago

Harshitmishra001 commented 3 weeks ago

I have created a updated Scrapper.py

-Custom Exception: Introducing a ScrapingError exception class allows you to handle errors specifically related to the scraping process.

-Explicit Timeouts: Using LOAD_MORE_TIMEOUT and SCRAPE_TIMEOUT makes the code more readable and easier to adjust the wait times for different operations.

-Error Handling in main: The main function now handles the ScrapingError exception, allowing the script to gracefully skip pages that encounter scraping issues.

-Constant Use: Using constants like MAX_PAGES improves readability and maintainability.

Harshitmishra001 commented 3 weeks ago

@sanjay-kv can you check this😊

github-actions[bot] commented 3 weeks ago

Hello @Harshitmishra001! Your issue #161 has been closed. Thank you for your contribution!