search

mikepqr / real-estate-scrape

Get the estimated value of a property from Redfin and Zillow
21 stars 0 forks source link

GitHub Action failing #5

Closed SamAtwell closed 1 year ago

SamAtwell commented 1 year ago

Hey,

I've been using the example repo you have for this scraping action and it looks like the GitHub action has been failing for the last few months. I am not a developer but from what I can see, it looks like the XPATHs changed for Redfin.

It is currently this: "xpath": "//div[@class='statsValue']//span/text()", but I think should instead be: "xpath": "//div[@class='statsValue']//span/text()",

Zillow also doesn't seem to be working but I can't see why it wouldn't.

If you are able to, can you take a look at this and update the Python package?

mikepqr commented 1 year ago

I assume you're talking about the actions on https://github.com/mikepqr/real-estate-scrape-eg/?

If so, there were two problems. One is fixed. The other cannot be fixed.

The second problem is not possible to fix and is out of scope for the tool, so I'm going to close this issue.

If you're seeing other errors then please post back with the exact errors and/or a link to the address you're trying to scrape.

SamAtwell commented 1 year ago

Hey,

Yep, I saw the error with the API key and updated that. But even after that, I am still getting an error saying "Could not find @.='statsValue']//span/text()" in url=''" in my GitHub Actions log. Which is what caused me to think that the XPATH needs to be changed.

------- Original Message ------- On Monday, September 25th, 2023 at 11:14 AM, Mike Lee Williams @.***> wrote:

I assume you're talking about the actions on https://github.com/mikepqr/real-estate-scrape-eg/?

If so, there were two problems. One is fixed. The other cannot be fixed.

-

the scraper api key that repo used had expired. That caused scraping both sites to fail with Unauthorized request, please make sure your API key is valid, (see e.g. https://github.com/mikepqr/real-estate-scrape-eg/actions/runs/6211714221/job/16861182019#step:5:1). This was fixed about a week ago. Zillow scraping is now working (see https://github.com/mikepqr/real-estate-scrape-eg/actions/runs/6295128552/job/17087975342#step:5:53 and https://github.com/mikepqr/real-estate-scrape-eg/blob/main/data.csv#L1097).

-

even with a working key, scraping Redfin for the particular address used as an example on https://github.com/mikepqr/real-estate-scrape-eg/ is broken. It's not because the xpath has changed. It's because that property is currently for sale on the market, which means there is no estimate on the page (see "Our gears are turning, but we don’t have enough information to generate an accurate estimate at this time. Learn More").

The second problem is not possible to fix and is out of scope for the tool, so I'm going to close this issue.

If you're seeing other errors then please post back with the exact errors and/or a link to the address you're trying to scrape.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

mlw0 commented 1 year ago

That xpath works fine for me on my home address. Not enough details in your question to investigate further. I'm going to leave this closed absent sufficient details.