COS420-Fall23 / Team-D

This is Team D's project repository. Shall be tackling the problem of helping graduates/students find a reliable job by cutting down the search method or type that other sites use.
3 stars 2 forks source link

Add scraping program #186

Closed ngauthier404 closed 7 months ago

ngauthier404 commented 7 months ago

This PR does a few things:

  1. Adds the URL parameter to the React app to match the data types I am using on Firestore
  2. Adds the Python scraper I used to update the Firestore.

The scraper program currently takes some setup to run and I haven't gotten around to updating the README to include the instructions, but it works enough to get new listings up on Firestore that will help us finish the project.

I will be back with another PR in a few days to do that.