search

Recode-Hive / Scrape-ML

For new data generation Semi-supervised-sequence-learning-Project we have writtern a python script to fetch📊, data from the 💻, imdb website 🌐 and converted into txt files.
https://scrape-ml.streamlit.app/
MIT License
80 stars 117 forks source link

Implementing Scheduled Scraping #178

Closed ShaikArshidBanu closed 5 days ago

ShaikArshidBanu commented 3 weeks ago

This PR closes issue #129 ,the suggested changes are implemented in this PR kindly close the PR #176 .

Description: Add functionality to the IMDb Movie Review Scraper to run at scheduled intervals automatically. This feature will ensure that the dataset is updated regularly without manual intervention.

Feature Details:

Suggestion: Implement a scheduling feature using Python's schedule library or system cron jobs to automate the scraping process. Benefit: Keeps the dataset updated with the latest reviews regularly and reduces the need for manual execution.

Expected Outcome: The scraper will run automatically at the scheduled intervals, ensuring the dataset is consistently updated with new reviews.

Additional Context: This feature is crucial for users who require up-to-date review data for analysis and research without the need for manual intervention. The implementation will be flexible to allow users to specify the desired interval for scraping.

@sanjay-kv Kindly review the PR.

github-actions[bot] commented 5 days ago

This PR has been automatically closed due to inactivity from the owner for 15 days.