search

Fitzclutchington / order_tracker

0 stars 0 forks source link

Scrape data from Used Safe Site #7

Closed Fitzclutchington closed 3 days ago

Fitzclutchington commented 6 days ago

Scrape and process safes from this url: https://empiresafe.com/pre-owned/all/

Create a CSV that can uploaded to the DB

Post processing is mostly extracting the individual dimensions from the dimension string.

Fitzclutchington commented 5 days ago

Not blocked per se, but current issue is that using requests to get the site data only returns the <head> element. My trusty advisor (chatGPT) mentioned this may be due to the site being rendered with javascript. A solution there is to use seleneium. Will look into this next.

Fitzclutchington commented 3 days ago

A few data issues, some weights/ capacities are empty when they shouldn't be, but we have enough data coverage to handle. I'm also sick of data scraping for the moment.