This repository contains the web scraping component of the Career Craft project, designed to collect job listings from various company websites.
The Career Craft Scrapper is a Node.js application that automates the process of gathering job postings from different company career pages. It currently supports scraping from:
Before running the scrapper, ensure you have the following installed:
Clone the repository:
git clone https://github.com/The-Enthusiast-404/career-craft-scrapper.git
cd career-craft-scrapper
Install dependencies:
npm install
The scrapper configuration is stored in config.js
. You can modify this file to add or update scraping targets and selectors.
To run the scrapper:
npm start
This will execute the main script (src/index.js
), which orchestrates the scraping process for all configured companies.
src/
: Contains the source code
index.js
: Main entry pointscrapers/
: Individual scraper modules for each companyutils/
: Utility functions and helpersconfig.js
: Configuration file for scraping targetspackage.json
: Project metadata and dependenciesContributions to improve the scrapper or add support for new companies are welcome. Please follow these steps:
git checkout -b feature/your-feature-name
)git commit -am 'Add some feature'
)git push origin feature/your-feature-name
)This project is licensed under the MIT License.