danieldotnl / ha-multiscrape

Home Assistant custom component for scraping (html, xml or json) multiple values (from a single HTTP request) with a separate sensor/attribute for each value. Support for (login) form-submit functionality.
MIT License
289 stars 16 forks source link

Resource offline #65

Closed theOrakle closed 2 years ago

theOrakle commented 3 years ago

Is your feature request related to a problem? Please describe. I have an IOT that is not online but every once in a while.

Describe the solution you'd like I would like the capability to quieting logs when the resource is offline and use the on_error to still have the entities default to a value.

Describe alternatives you've considered On_error doesn't take effect when the resource in unreachable

Additional context None

txespirito commented 2 years ago

I have the same problem as theOrakle, but with a peculiarity. Like theOrakle, on_error doesn't take effect when the resource in unreachable, but in my case if I do different scrape scrape queries for different sensors it is always available, but if I group the queries with multiscrape the sensors are not available intermittently. In the image I put an example with the two forms.

danieldotnl commented 2 years ago

Could you guys check the behavior on release v6.2.0?

theOrakle commented 2 years ago

I still see this in the logs:

2022-04-24 21:25:31 ERROR (MainThread) [custom_components.multiscrape.http] Scraper_noname_0 # Error executing get request to url: http://bbq.bla-bla.local.
 Error message:
 ConnectError('All connection attempts failed')

There is a possibility I do not the on error configured correctly, but given how easy it is, I would be surprised:

    on_error:
      log: false
      value: default
      default: 0
StevenBrs commented 2 years ago

Hi All,

I have the same issue reading out the values of my solar panels. The inverter is only online when powered by the sun. I tried adding on_error, but this doesn't work and the entities are marked as "Unavailable". Version I'm running is 6.2.2

multiscrape:
  - name: Solis WiFi Scraper
    resource: ******
    authentication: basic
    username: ******
    password: ******
    scan_interval: 60
    sensor:
      - unique_id: solis_1_webdata_now_p
        name: Current Power (W)
        icon: mdi:solar-power
        unit_of_measurement: "W"
        select: "script:nth-of-type(2)"
        value_template: "{{ (value | regex_findall('webdata_now_p = \"(.*)\";', true))[0]|int }}"
        force_update: true
        on_error:
          value: default
          default: 0
        attributes:
        - name: "Solis S/N"
          value_template: "{{ (value | regex_findall('webdata_sn = \"(.*)\";', true))[0] }}"
      - unique_id: solis_1_webdata_today_e
        name: Yield Today (kWh)
        icon: mdi:solar-power
        unit_of_measurement: "kWh"
        state_class: measurement
        device_class: energy
        select: "script:nth-of-type(2)"
        value_template: "{{ (value | regex_findall('webdata_today_e = \"(.*)\";', true))[0]|float }}"
        force_update: true
        on_error:
          value: none
        attributes:
        - name: "Solis S/N"
          value_template: "{{ (value | regex_findall('webdata_sn = \"(.*)\";', true))[0] }}"
        - name: last_reset
          value_template: "{{ now().strftime('%Y-%m-%dT00:00:00') }}"

Maybe I can work around this by passing through nginx as a proxy, but it would be nice to have the on_error parameter available.

And also: Thx @danieldotnl for your work!

danieldotnl commented 2 years ago

Could you guys check out pre-v6.3.0?

danieldotnl commented 2 years ago

Please create a new issue if the problems still exist.

StevenBrs commented 2 years ago

Hi Daniel,

Sorry for the late reply.

My last check was that it was working correctly, but shortly after I switched to another method of reading the data and publish it into MQTT, so I’m not able to test thoroughly.

Thanks for your commitment and the change though! I’m sure it will come in handy someday.

Best regards

Steven

From: Daniel @.> Sent: dinsdag 3 mei 2022 21:42 To: danieldotnl/ha-multiscrape @.> Cc: StevenBrs @.>; Comment @.> Subject: Re: [danieldotnl/ha-multiscrape] Resource offline (Issue #65)

Could you guys check out pre-v6.3.0?

— Reply to this email directly, view it on GitHub https://github.com/danieldotnl/ha-multiscrape/issues/65#issuecomment-1116492431 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ADT5FTAR2CB7OQOET7XMWVLVIF6PVANCNFSM5HRN2AEA . You are receiving this because you commented. https://github.com/notifications/beacon/ADT5FTEPBP5PVKW36RZPXGTVIF6PVA5CNFSM5HRN2AEKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOIKGFFDY.gif Message ID: @. @.> >