search

danieldotnl / ha-multiscrape

Home Assistant custom component for scraping (html, xml or json) multiple values (from a single HTTP request) with a separate sensor/attribute for each value. Support for (login) form-submit functionality.
MIT License
277 stars 14 forks source link

Client error '451 Unavailable For Legal Reasons' for url #364

Closed inancdev closed 3 weeks ago

inancdev commented 5 months ago

I am trying to scrape from flightradar24 to check if my plane landed or not. Using similar setup with Tasker on my Android device and it works however when I try same with service call on HA, I got 451 error for some reason.

CSS selector should be: "#tbl-datatable > tbody > tr.live.data-row > td.visible-xs.visible-sm > div.row.table-row-responsive > div.col-xs-3> div.row:nth-child(4)"

httpx.HTTPStatusError: Client error '451 Unavailable For Legal Reasons' for url 'https://www.flightradar24.com/data/flights/[INSERT-ANY-LIVE-FLIGHT-NUMBER-HERE-WITHOUT-BRACKETS]' For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/451

danieldotnl commented 3 weeks ago

Probably they have a mechanism to detect scraping and try to block that. Don't know if there are ways around it. Anyway, there is an excellent custom component for flightradar24. I would give that one a try.