JonathanWamsley / indeed-data-job-scraper

Scrapes indeed at data related urls jobs
0 stars 0 forks source link

Set up project structure and initial configurations #1

Closed JonathanWamsley closed 2 hours ago

JonathanWamsley commented 3 hours ago

Create the project directory structure to include initial configurations to streamline development, including directories, files, and basic setup for Github actions.

Task:

  1. create directory structure:

    • src/ for python scripts for scraping
    • data/ for storing data
    • docs/ for project documentation
    • tests/ for unit tests related to the project
    • .github/workflows - for github actions yaml files
  2. Setup basic project files

    • README.md - overview of the project, instructions for setup and usage
    • .gitignore
    • requirements.txt
    • config.yml - initial configuration file for scraping settings
  3. Initial github actions workflow

    • add a basic github actions workflow in .github/workflows/ci.yml to include:
    • test: for running pytest
    • format: code format check with black
    • lint: code linting using pylint
    1. Documentation link:
      • create a placeholder in the README.md linking to documentation information
JonathanWamsley commented 2 hours ago

Initial project created with github copilot workspace. It is extremely buggy and applying additional updates and fixing errors is not a streamlined process.