search

danieldotnl / ha-multiscrape

Home Assistant custom component for scraping (html, xml or json) multiple values (from a single HTTP request) with a separate sensor/attribute for each value. Support for (login) form-submit functionality.
MIT License
293 stars 16 forks source link

XML Parsed Error #261

Open Ricks88 opened 1 year ago

Ricks88 commented 1 year ago

Hello,

Not sure if this is a Issue/bug, but I am getting this new error: Deze fout is ontstaan door een aangepaste integratie.

Logger: py.warnings Source: custom_components/multiscrape/scraper.py:59 Integration: Multiscrape scraping component (documentation, issues) First occurred: 16:25:02 (2 occurrences) Last logged: 16:26:07

/usr/local/lib/python3.11/site-packages/bs4/builder/init.py:545: XMLParsedAsHTMLWarning: It looks like you're parsing an XML document using an HTML parser. If this really is an HTML document (maybe it's XHTML?), you can ignore or filter this warning. If it's XML, you should know that using an XML parser will be more reliable. To parse this document as XML, make sure you have the lxml package installed, and pass the keyword argument features="xml" into the BeautifulSoup constructor. warnings.warn(

My code is: `multiscrape:

I only have used BS4 as Package into AppDeamon, I dont understand why it is using BS4 in the multiscrape or how to use XML parser instead.

codyc1515 commented 1 year ago

Here is an example URL, which is JSON (so not either XML or HTML): https://www.metservice.com/publicData/webdata/warnings-service/urban/auckland

This error originated from a custom integration.

Logger: py.warnings
Source: custom_components/multiscrape/scraper.py:59
Integration: Multiscrape scraping component (documentation, issues)
First occurred: 09:30:42 (3 occurrences)
Last logged: 09:57:32

/usr/local/lib/python3.11/site-packages/bs4/builder/__init__.py:545: XMLParsedAsHTMLWarning: It looks like you're parsing an XML document using an HTML parser. If this really is an HTML document (maybe it's XHTML?), you can ignore or filter this warning. If it's XML, you should know that using an XML parser will be more reliable. To parse this document as XML, make sure you have the lxml package installed, and pass the keyword argument `features="xml"` into the BeautifulSoup constructor. warnings.warn(
diamant-x commented 8 months ago

Same here, and same issues as https://github.com/danieldotnl/ha-multiscrape/issues/220 The suggested solution didn't solve it. Currently is flooding the HA logs for every time it runs, even after setting parser: 'lxml'.

There should be a way to filter out this specific warning. Or maybe reduce the criticality to WARN or INFO so that it can be filtered by native log support? https://community.home-assistant.io/t/how-to-exclude-a-specific-warning-from-log/38761/13?u=diamant-x

Thanks.

stephan1827 commented 7 months ago

Is there any solution for this?

Ricks88 commented 7 months ago

Is there any solution for this?

@diamant-x @stephan1827

I just use this in the logger:

`

logger: default: warning

filters: homeassistant:

diamant-x commented 7 months ago

homeassistant.loader:

Wonderful, will try, thanks! PD: How did you identified this was the filter component to use?

danieldotnl commented 7 months ago

Nice workaround! It would help me a lot if someone can provide me a configuration to reproduce this. https://hp-officejet.local doesn't work for me :)

Ricks88 commented 7 months ago

homeassistant.loader:

Wonderful, will try, thanks! PD: How did you identified this was the filter component to use?

I did not, i just looked at the logs and copy the first words of the notification i do not want to see anymore. Maybe just only

Ricks88 commented 7 months ago

Nice workaround! It would help me a lot if someone can provide me a configuration to reproduce this. https://hp-officejet.local doesn't work for me :)

Use the IP of your own printer..

OR

Just start using my instant_ink_print_plan yaml

https://github.com/Ricks88/HASS_HPInstantInk/tree/main

No more need of ping sensor and multiscrape or any XML errors.

stephan1827 commented 7 months ago

@Ricks88 I tried the log filter but I still get the log entries. My entry looks like this.

logger:
    default: warning

    filters:
        homeassistant:
            - "Error doing job: Task exception was never retrieved"

        homeassistant.loader:
            - "We found a custom integration"
            - "XMLParsedAsHTMLWarning"
            - "It looks like you're parsing an XML document using an HTML parser"

Is there something wrong in the formating?

Ricks88 commented 7 months ago

@Ricks88 I tried the log filter but I still get the log entries. My entry looks like this.

Is there something wrong in the formating?

@stephan1827 @danieldotnl

Yes, sorry!

Correct config:

logger:
  default: warning

  filters:
    homeassistant:
      - "Error doing job: Task exception was never retrieved"

    homeassistant.loader:
      - "We found a custom integration"

    pychromecast.socket_client:
      - "Failed to connect to service ServiceInfo"
      - "Error communicating with socket, resetting connection"

    py.warnings:
      # Ignore the BeautifulSoup XML parser error with Multiscrape and HP Printer XML file.
      - "XMLParsedAsHTMLWarning"
stephan1827 commented 7 months ago

@Ricks88 Thx that worked

danieldotnl commented 7 months ago

I reproduced this issue and setting the parser to xml seems to solve it for me. Could you give it a try and confirm?

stephan1827 commented 7 months ago

Just tried it and the error is still there. Do I have to change the parser option, too. Mine is set to lxml.

danieldotnl commented 7 months ago

Try setting it to 'xml' and let me know pls

stephan1827 commented 7 months ago

When I do that I get

Consider using debug logging and log_response for further investigation. 2024-04-03 10:04:25.425 ERROR (MainThread) [custom_components.multiscrape.sensor] Scraper_noname_0 # Piko yield total (Wh) # Unable to scrape data: Could not find a tag for given selector

This is my configuration

# -------------------------------------------------------
# PIKO inverter
multiscrape:
  - resource: http://192.168.30.82/all.xml
    scan_interval: 15 # Kann auch grobmaschiger abgefragt werden z.B. 60s
    parser: xml
    sensor:
      - unique_id: kostal_piko_ac_power
        name: "Piko AC Power"
        select: "measurement[type='AC_Power']"
        attribute: "value"
        unit_of_measurement: "W"
        on_error:
          log: 'info'
          value: "default"
          default: 0.0
      - unique_id: kostal_piko_yield_total_Wh
        name: "Piko yield total (Wh)"
        select: "yieldvalue"
        attribute: "value"
        unit_of_measurement: "Wh"
        state_class: total_increasing
        device_class: energy

and here is the xml data from the inverter

This XML file does not appear to have any style information associated with it. The document tree is shown below.
<root>
<Device Name="PIKO 2.5-1 MP plus" Type="Inverter" Platform="Net16" HmiPlatform="HMI17" NominalPower="2500" UserPowerLimit="nan" CountryPowerLimit="nan" Serial="763166EG006884130005" OEMSerial="10351313" BusAddress="1" NetBiosName="INV006884130005" WebPortal="PIKO Solar Portal" ManufacturerURL="kostal-solar-electric.com" IpAddress="192.168.30.82" DateTime="2024-04-03T09:04:21" MilliSeconds="207">
<Siblings/>
<CanOpen Status="not started"/>
<Measurements>
<Measurement Value="223.8" Unit="V" Type="AC_Voltage"/>
<Measurement Value="1.213" Unit="A" Type="AC_Current"/>
<Measurement Value="271.8" Unit="W" Type="AC_Power"/>
<Measurement Value="273.6" Unit="W" Type="AC_Power_fast"/>
<Measurement Value="49.991" Unit="Hz" Type="AC_Frequency"/>
<Measurement Value="269.0" Unit="V" Type="DC_Voltage"/>
<Measurement Value="1.114" Unit="A" Type="DC_Current"/>
<Measurement Value="330.0" Unit="V" Type="LINK_Voltage"/>
<Measurement Unit="W" Type="GridPower"/>
<Measurement Unit="W" Type="GridConsumedPower"/>
<Measurement Unit="W" Type="GridInjectedPower"/>
<Measurement Unit="W" Type="OwnConsumedPower"/>
<Measurement Value="100.0" Unit="%" Type="Derating"/>
</Measurements>
<Events>
<Event Id="37" Message="ENS Line connection wrong" Severity="Error" Type="User" Start="2024-04-02T10:41:52" End="2024-04-02T10:44:07"/>
<Event Id="40" Message="ENS Grid Voltage too low L1" Severity="Error" Type="User" Start="2024-04-02T10:41:52" End="2024-04-02T10:44:07"/>
<Event Id="1" Message="ENS Grid Frequency too low" Severity="Error" Type="User" Start="2023-12-21T10:28:54" End="2023-12-21T10:28:55"/>
<Event Id="37" Message="ENS Line connection wrong" Severity="Error" Type="User" Start="2023-12-21T10:28:54" End="2023-12-21T10:31:25"/>
<Event Id="40" Message="ENS Grid Voltage too low L1" Severity="Error" Type="User" Start="2023-10-24T13:22:44" End="2023-10-24T13:22:45"/>
<Event Id="37" Message="ENS Line connection wrong" Severity="Error" Type="User" Start="2023-10-24T13:22:44" End="2023-10-24T13:22:48"/>
<Event Id="40" Message="ENS Grid Voltage too low L1" Severity="Error" Type="User" Start="2023-08-16T11:25:23" End="2023-08-16T11:25:24"/>
<Event Id="37" Message="ENS Line connection wrong" Severity="Error" Type="User" Start="2023-08-16T11:25:23" End="2023-08-16T11:27:38"/>
<Event Id="40" Message="ENS Grid Voltage too low L1" Severity="Error" Type="User" Start="2023-08-16T11:14:41" End="2023-08-16T11:14:41"/>
<Event Id="37" Message="ENS Line connection wrong" Severity="Error" Type="User" Start="2023-08-16T11:14:41" End="2023-08-16T11:22:47"/>
<Event Id="37" Message="ENS Line connection wrong" Severity="Error" Type="User" Start="2023-07-30T10:15:00" End="2023-07-30T10:15:11"/>
<Event Id="40" Message="ENS Grid Voltage too low L1" Severity="Error" Type="User" Start="2023-07-30T10:15:00" End="2023-07-30T10:15:11"/>
<Event Id="37" Message="ENS Line connection wrong" Severity="Error" Type="User" Start="2023-04-28T14:06:01" End="2023-04-28T14:06:34"/>
<Event Id="40" Message="ENS Grid Voltage too low L1" Severity="Error" Type="User" Start="2023-04-28T14:06:01" End="2023-04-28T14:06:34"/>
<Event Id="33" Message="PU SW version incompatible" Severity="Info" Type="User" Start="2023-02-25T08:50:39" End="2023-02-25T09:21:43"/>
<Event Id="85" Message="PU OEM variant mismatch" Severity="Error" Type="User" Start="2023-02-25T08:50:19" End="2023-02-25T09:23:23"/>
<Event Id="38" Message="ENS Reading countrycode failed" Severity="Error" Type="User" Start="2023-02-25T08:50:18" End="2023-02-25T09:23:23"/>
<Event Id="85" Message="PU OEM variant mismatch" Severity="Error" Type="User" Start="2023-02-25T08:45:14" End="2023-02-25T08:50:10"/>
<Event Id="33" Message="PU SW version incompatible" Severity="Info" Type="User" Start="2023-02-25T08:45:32" End="2023-02-25T08:48:28"/>
<Event Id="38" Message="ENS Reading countrycode failed" Severity="Error" Type="User" Start="2023-02-25T08:45:13" End="2023-02-25T08:50:10"/>
<Event Id="33" Message="PU SW version incompatible" Severity="Info" Type="User" Start="2023-02-25T08:07:52" End="2023-02-25T08:45:03"/>
<Event Id="85" Message="PU OEM variant mismatch" Severity="Error" Type="User" Start="2023-02-25T08:07:33" End="2023-02-25T08:45:03"/>
<Event Id="38" Message="ENS Reading countrycode failed" Severity="Error" Type="User" Start="2023-02-25T08:07:32" End="2023-02-25T08:45:03"/>
<Event Id="40" Message="ENS Grid Voltage too low L1" Severity="Error" Type="User" Start="2022-11-27T11:44:56" End="2022-11-27T11:44:56"/>
<Event Id="37" Message="ENS Line connection wrong" Severity="Error" Type="User" Start="2022-11-27T11:44:56" End="2022-11-27T11:45:13"/>
<Event Id="40" Message="ENS Grid Voltage too low L1" Severity="Error" Type="User" Start="2022-11-19T09:55:22" End="2022-11-19T09:55:23"/>
<Event Id="37" Message="ENS Line connection wrong" Severity="Error" Type="User" Start="2022-11-19T09:55:22" End="2022-11-19T09:55:36"/>
<Event Id="37" Message="ENS Line connection wrong" Severity="Error" Type="User" Start="2022-10-23T09:05:39" End="2022-10-23T09:05:55"/>
<Event Id="40" Message="ENS Grid Voltage too low L1" Severity="Error" Type="User" Start="2022-10-23T09:05:39" End="2022-10-23T09:05:55"/>
<Event Id="11" Message="ENS DC Offset too high" Severity="Error" Type="User" Start="2022-10-22T11:08:59" End="2022-10-22T11:32:20"/>
<Event Id="12" Message="ENS residual current too high" Severity="Error" Type="User" Start="2022-10-22T11:08:59" End="2022-10-22T11:32:20"/>
<Event Id="122" Message="PU Battery manager values missing" Severity="Warning" Type="User" Start="2022-10-22T11:08:59" End="2022-10-22T11:32:20"/>
<Event Id="34" Message="ENS Remote Tripping" Severity="Warning" Type="User" Start="2022-10-22T11:08:59" End="2022-10-22T11:32:20"/>
<Event Id="82" Message="DRM0 opencircuit" Severity="Error" Type="User" Start="2022-10-22T11:08:59" End="2022-10-22T11:32:20"/>
<Event Id="121" Message="Check grid support config" Severity="Error" Type="User" Start="2022-10-22T11:08:59" End="2022-10-22T11:32:20"/>
<Event Id="43" Message="ENS Grid Voltage too high L1" Severity="Error" Type="User" Start="2022-10-22T11:08:59" End="2022-10-22T11:32:20"/>
<Event Id="5" Message="ENS Grid Voltage too low for reconnect" Severity="Error" Type="User" Start="2022-10-22T11:08:59" End="2022-10-22T11:32:20"/>
<Event Id="6" Message="ENS Grid Voltage too high for reconnect" Severity="Error" Type="User" Start="2022-10-22T11:08:59" End="2022-10-22T11:32:20"/>
<Event Id="7" Message="ENS Grid AVG Voltage too low" Severity="Error" Type="User" Start="2022-10-22T11:08:59" End="2022-10-22T11:32:20"/>
<Event Id="1" Message="ENS Grid Frequency too low" Severity="Error" Type="User" Start="2022-10-22T11:08:59" End="2022-10-22T11:32:20"/>
<Event Id="2" Message="ENS Grid Frequency too high" Severity="Error" Type="User" Start="2022-10-22T11:08:59" End="2022-10-22T11:32:20"/>
<Event Id="37" Message="ENS Line connection wrong" Severity="Error" Type="User" Start="2022-08-03T04:31:25" End="2022-08-03T04:31:52"/>
<Event Id="40" Message="ENS Grid Voltage too low L1" Severity="Error" Type="User" Start="2022-08-03T04:31:25" End="2022-08-03T04:31:52"/>
<Event Id="40" Message="ENS Grid Voltage too low L1" Severity="Error" Type="User" Start="2022-06-10T09:49:07" End="2022-06-10T09:49:07"/>
<Event Id="37" Message="ENS Line connection wrong" Severity="Error" Type="User" Start="2022-06-10T09:49:07" End="2022-06-10T09:54:30"/>
<Event Id="37" Message="ENS Line connection wrong" Severity="Error" Type="User" Start="2022-05-28T08:09:48" End="2022-05-28T08:22:23"/>
<Event Id="40" Message="ENS Grid Voltage too low L1" Severity="Error" Type="User" Start="2022-05-28T08:09:48" End="2022-05-28T08:22:23"/>
<Event Id="5" Message="ENS Grid Voltage too low for reconnect" Severity="Error" Type="User" Start="2022-03-05T08:02:47" End="2022-03-05T08:02:47"/>
<Event Id="40" Message="ENS Grid Voltage too low L1" Severity="Error" Type="User" Start="2022-03-05T08:02:47" End="2022-03-05T08:02:47"/>
<Event Id="37" Message="ENS Line connection wrong" Severity="Error" Type="User" Start="2022-03-05T08:02:47" End="2022-03-05T09:03:31"/>
<Event Id="37" Message="ENS Line connection wrong" Severity="Error" Type="User" Start="2022-03-05T08:00:44" End="2022-03-05T08:02:41"/>
<Event Id="40" Message="ENS Grid Voltage too low L1" Severity="Error" Type="User" Start="2022-03-05T08:00:44" End="2022-03-05T08:02:41"/>
<Event Id="37" Message="ENS Line connection wrong" Severity="Error" Type="User" Start="2022-03-02T17:03:54" End="2022-03-02T17:04:24"/>
<Event Id="40" Message="ENS Grid Voltage too low L1" Severity="Error" Type="User" Start="2022-03-02T17:03:54" End="2022-03-02T17:04:24"/>
<Event Id="37" Message="ENS Line connection wrong" Severity="Error" Type="User" Start="2021-08-13T08:59:56" End="2021-08-13T09:00:46"/>
<Event Id="40" Message="ENS Grid Voltage too low L1" Severity="Error" Type="User" Start="2021-08-13T08:59:56" End="2021-08-13T09:00:46"/>
<Event Id="37" Message="ENS Line connection wrong" Severity="Error" Type="User" Start="2021-08-12T07:53:20" End="2021-08-12T07:54:59"/>
<Event Id="40" Message="ENS Grid Voltage too low L1" Severity="Error" Type="User" Start="2021-08-12T07:53:20" End="2021-08-12T07:54:59"/>
<Event Id="37" Message="ENS Line connection wrong" Severity="Error" Type="User" Start="2021-06-16T07:10:19" End="2021-06-16T07:10:36"/>
<Event Id="40" Message="ENS Grid Voltage too low L1" Severity="Error" Type="User" Start="2021-06-16T07:10:19" End="2021-06-16T07:10:36"/>
</Events>
<Yields>
<Yield Type="Produced" Slot="Total" Unit="Wh">
<YieldValue Value="8946845" TimeStamp="2020-09-25T13:20:00"/>
</Yield>
</Yields>
<Versions>
<Software Device="HMI" Name="BFAPI STM32F4" Version="2.8.0"/>
<Software Device="HMI" Name="FBL" Version="1.4.0"/>
<Software Device="HMI" Name="APP" Version="3.18.0"/>
<Software Device="HMI" Name="OEM PIKO 2.5-1 MP p" Version="1.0.1"/>
<Software Device="PU" Name="BFAPI SAFE STM32F4" Version="2.8.0"/>
<Software Device="PU" Name="FBL" Version="2.0.3"/>
<Software Device="PU" Name="APP" Version="4.7.0"/>
<Software Device="PU" Name="PAR default" Version="24.0.24"/>
<Software Device="PU" Name="OEM VAR_7_Kostal" Version="1.0.7"/>
<Software Device="ENS1" Name="APP" Version="1.35.0"/>
<Hardware Device="HMI" Version="1"/>
<Hardware Device="PU" Version="5"/>
</Versions>
<State Value="Standby"/>
<Tasks/>
<Statistics>
<Statistic Name="Common">
<Value Label="Connected%20To%20Grid">71832415</Value>
<Value Label="Active">34706282</Value>
</Statistic>
<Statistic Name="Power">
<Value Label="0-50W">5212456</Value>
<Value Label="50-100W">3660647</Value>
<Value Label="100-200W">4569596</Value>
<Value Label="200-500W">6926171</Value>
<Value Label="500-1000W">5500647</Value>
<Value Label="1000-2000W">5873724</Value>
<Value Label="2000-5000W">2963041</Value>
<Value Label="5000-10000W">0</Value>
</Statistic>
<Statistic Name="AC%20Voltage%201">
<Value Label="%3C200V">5002</Value>
<Value Label="200-210V">5</Value>
<Value Label="210-220V">4458</Value>
<Value Label="220-230V">60011127</Value>
<Value Label="230-240V">11806666</Value>
<Value Label="240-250V">618</Value>
<Value Label="250-255V">0</Value>
<Value Label="255-260V">0</Value>
<Value Label="260-265V">0</Value>
<Value Label="%3E265V">0</Value>
</Statistic>
<Statistic Name="PV%20Voltage%201">
<Value Label="%3C100V">0</Value>
<Value Label="100-200V">0</Value>
<Value Label="200-300V">0</Value>
<Value Label="300-400V">0</Value>
<Value Label="400-500V">0</Value>
<Value Label="500-600V">0</Value>
<Value Label="600-700V">0</Value>
<Value Label="700-800V">0</Value>
<Value Label="800-850V">0</Value>
<Value Label="850-900V">0</Value>
<Value Label="900-950V">0</Value>
<Value Label="950-1000V">0</Value>
<Value Label="%3E1000V">0</Value>
</Statistic>
<Statistic Name="Derating">
<Value Label="Temperature">0</Value>
<Value Label="External">50</Value>
<Value Label="P(f)">751</Value>
<Value Label="Startup">4318</Value>
<Value Label="UserLimit">0</Value>
<Value Label="Hardware%20Limit">0</Value>
<Value Label="Frequency%20TooHigh">0</Value>
</Statistic>
<Statistic Name="Temp%20Ambient">
<Value Label="%3C-20%C2%B0C">0</Value>
<Value Label="-20-0%C2%B0C">0</Value>
<Value Label="0-20%C2%B0C">9671993</Value>
<Value Label="20-40%C2%B0C">57466469</Value>
<Value Label="40-60%C2%B0C">4046873</Value>
<Value Label="60-80%C2%B0C">18449</Value>
<Value Label="80-100%C2%B0C">220</Value>
<Value Label="%3E100%C2%B0C">628411</Value>
</Statistic>
<Statistic Name="Temp%20T1">
<Value Label="%3C-20%C2%B0C">0</Value>
<Value Label="-20-0%C2%B0C">0</Value>
<Value Label="0-20%C2%B0C">13897338</Value>
<Value Label="20-40%C2%B0C">44713372</Value>
<Value Label="40-60%C2%B0C">13217167</Value>
<Value Label="60-80%C2%B0C">0</Value>
<Value Label="80-100%C2%B0C">0</Value>
<Value Label="%3E100%C2%B0C">0</Value>
</Statistic>
<Statistic Name="Temp%20T2">
<Value Label="%3C-20%C2%B0C">0</Value>
<Value Label="-20-0%C2%B0C">0</Value>
<Value Label="0-20%C2%B0C">13897338</Value>
<Value Label="20-40%C2%B0C">44713372</Value>
<Value Label="40-60%C2%B0C">13217167</Value>
<Value Label="60-80%C2%B0C">0</Value>
<Value Label="80-100%C2%B0C">0</Value>
<Value Label="%3E100%C2%B0C">0</Value>
</Statistic>
<Statistic Name="Temp%20T3">
<Value Label="%3C-20%C2%B0C">0</Value>
<Value Label="-20-0%C2%B0C">0</Value>
<Value Label="0-20%C2%B0C">13897338</Value>
<Value Label="20-40%C2%B0C">44713372</Value>
<Value Label="40-60%C2%B0C">13217167</Value>
<Value Label="60-80%C2%B0C">0</Value>
<Value Label="80-100%C2%B0C">0</Value>
<Value Label="%3E100%C2%B0C">0</Value>
</Statistic>
<Statistic Name="Temp%20T4">
<Value Label="%3C-20%C2%B0C">0</Value>
<Value Label="-20-0%C2%B0C">0</Value>
<Value Label="0-20%C2%B0C">13897338</Value>
<Value Label="20-40%C2%B0C">44713372</Value>
<Value Label="40-60%C2%B0C">13217167</Value>
<Value Label="60-80%C2%B0C">0</Value>
<Value Label="80-100%C2%B0C">0</Value>
<Value Label="%3E100%C2%B0C">0</Value>
</Statistic>
<Statistic Name="PU%20Tmax">
<Value Label="%3C-20%C2%B0C">0</Value>
<Value Label="-20-0%C2%B0C">0</Value>
<Value Label="0-20%C2%B0C">13897465</Value>
<Value Label="20-40%C2%B0C">38573348</Value>
<Value Label="40-60%C2%B0C">17063338</Value>
<Value Label="60-80%C2%B0C">2293726</Value>
<Value Label="80-100%C2%B0C">0</Value>
<Value Label="%3E100%C2%B0C">0</Value>
</Statistic>
</Statistics>
<Settings>
<PowerLimit Value="2500" Valid="false"/>
<DateTimeFormat Date="dd.mm.yyyy" Time="24h"/>
</Settings>
<ReactivePower>
<Curve Type="cosPhi = 1" XUnit="‰" YUnit="">
<Point X="0" Y="1000"/>
<Point X="1000" Y="1000"/>
</Curve>
<Parameters>
<Parameter Name="PowerFactor" Value="0.0" Unit="%"/>
<Parameter Name="Q fix" Value="0.0" Unit="var"/>
<Parameter Name="φ PLockIn" Value="10.0" Unit="%"/>
<Parameter Name="φ PLockOut" Value="10.0" Unit="%"/>
<Parameter Name="Q dynamik" Value="3.3" Unit="s"/>
</Parameters>
</ReactivePower>
<GridParameters CountryCode="4901" CountryName="D AR4105:2019">
<GridParameter ID="24" SortID="20" Name="t Reconnect" Value="60.00" Min="0.00" Max="1200.00" Unit="s" Var="true" Default="60.00"/>
<GridParameter ID="92" SortID="22" Name="t Startup" Value="60.00" Min="0.00" Max="1200.00" Unit="s" Var="true" Default="60.00"/>
<GridParameter ID="25" SortID="30" Name="Recon Grad" Value="10.00" Min="6.00" Max="3000.00" Unit="%" Var="true" Default="10.00"/>
<GridParameter ID="83" SortID="31" Name="StartupGrad" Value="100.00" Min="0.00" Max="3000.00" Unit="%" Var="true" Default="100.00"/>
<GridParameter ID="93" SortID="32" Name="Extern Grad" Value="39.40" Min="0.00" Max="30000.00" Unit="%" Var="true" Default="39.40"/>
<GridParameter ID="0" SortID="90" Name="U n" Value="230.00" Min="100.00" Max="240.00" Unit="V" Var="true" Default="230.00"/>
<GridParameter ID="10" SortID="100" Name="U > recon" Value="110.00" Min="100.00" Max="135.00" Unit="%" Var="true" Default="110.00"/>
<GridParameter ID="49" SortID="130" Name="U >>" Value="125.00" Min="100.00" Max="135.00" Unit="%" Var="true" Default="125.00"/>
<GridParameter ID="50" SortID="140" Name="t(U >>)" Value="0.20" Min="0.00" Max="180.00" Unit="s" Var="true" Default="0.20"/>
<GridParameter ID="9" SortID="150" Name="U < recon" Value="85.00" Min="5.00" Max="100.00" Unit="%" Var="true" Default="85.00"/>
<GridParameter ID="1" SortID="160" Name="U <" Value="80.00" Min="5.00" Max="100.00" Unit="%" Var="true" Default="80.00"/>
<GridParameter ID="2" SortID="170" Name="t(U <)" Value="3.00" Min="0.00" Max="180.00" Unit="s" Var="true" Default="3.00"/>
<GridParameter ID="51" SortID="180" Name="U <<" Value="45.00" Min="5.00" Max="100.00" Unit="%" Var="true" Default="45.00"/>
<GridParameter ID="52" SortID="190" Name="t(U <<)" Value="0.40" Min="0.00" Max="180.00" Unit="s" Var="true" Default="0.40"/>
<GridParameter ID="7" SortID="220" Name="U >" Value="110.00" Min="110.00" Max="115.00" Unit="%" Var="true" Default="110.00"/>
<GridParameter ID="8" SortID="230" Name="t(U >)" Value="600.00" Min="1.00" Max="603.00" Unit="s" Var="true" Default="600.00"/>
<GridParameter ID="84" SortID="270" Name="U HVRT" Value="115.00" Min="100.00" Max="135.00" Unit="%" Var="true" Default="115.00"/>
<GridParameter ID="85" SortID="275" Name="U LVRT" Value="80.00" Min="5.00" Max="100.00" Unit="%" Var="true" Default="80.00"/>
<GridParameter ID="11" SortID="320" Name="f n" Value="50.00" Min="50.00" Max="60.00" Unit="Hz" Var="false" Default="50.00"/>
<GridParameter ID="22" SortID="410" Name="f > recon" Value="50.10" Min="47.50" Max="51.50" Unit="Hz" Var="true" Default="50.10"/>
<GridParameter ID="14" SortID="420" Name="f >" Value="51.50" Min="50.00" Max="55.00" Unit="Hz" Var="true" Default="51.50"/>
<GridParameter ID="15" SortID="430" Name="t(f >)" Value="0.20" Min="0.05" Max="100.00" Unit="s" Var="true" Default="0.20"/>
<GridParameter ID="21" SortID="480" Name="f < recon" Value="47.50" Min="47.50" Max="51.50" Unit="Hz" Var="true" Default="47.50"/>
<GridParameter ID="12" SortID="490" Name="f <" Value="47.50" Min="45.00" Max="50.00" Unit="Hz" Var="true" Default="47.50"/>
<GridParameter ID="13" SortID="500" Name="t(f <)" Value="0.20" Min="0.05" Max="100.00" Unit="s" Var="true" Default="0.20"/>
<GridParameter ID="33" SortID="550" Name="I DCRelInom" Value="0.50" Min="0.10" Max="10.00" Unit="%" Var="true" Default="0.50"/>
<GridParameter ID="35" SortID="570" Name="t(IDC rel)" Value="5.00" Min="0.02" Max="5.00" Unit="s" Var="true" Default="5.00"/>
<GridParameter ID="26" SortID="600" Name="P(f)+ Grad" Value="40.00" Min="10.00" Max="1000.00" Unit="%/Hz" Var="true" Default="40.00"/>
<GridParameter ID="20" SortID="610" Name="P(f)+ Start" Value="50.20" Min="50.20" Max="50.50" Unit="Hz" Var="true" Default="50.20"/>
<GridParameter ID="62" SortID="630" Name="P(f)+ Exit" Value="50.20" Min="50.00" Max="50.20" Unit="Hz" Var="true" Default="50.20"/>
<GridParameter ID="74" SortID="650" Name="P(f) ExGrad" Value="10.00" Min="0.00" Max="100.00" Unit="%" Var="true" Default="10.00"/>
<GridParameter ID="86" SortID="660" Name="P(f)- Grad" Value="40.00" Min="16.60" Max="100.00" Unit="%/Hz" Var="true" Default="40.00"/>
<GridParameter ID="73" SortID="670" Name="P(f)- Exit" Value="49.80" Min="49.80" Max="50.00" Unit="Hz" Var="true" Default="49.80"/>
<GridParameter ID="70" SortID="680" Name="P(f)- Start" Value="49.80" Min="46.00" Max="50.00" Unit="Hz" Var="true" Default="49.80"/>
<GridParameter ID="65" SortID="800" Name="P(U) Modus" Value="0.00" Min="0.00" Max="2.00" Unit="" Var="true" Default="0.00"/>
<GridParameter ID="66" SortID="810" Name="P(U)+ Entry" Value="110.00" Min="0.00" Max="250.00" Unit="%" Var="true" Default="110.00"/>
<GridParameter ID="67" SortID="820" Name="P(U)+ End" Value="112.00" Min="0.00" Max="250.00" Unit="%" Var="true" Default="112.00"/>
<GridParameter ID="76" SortID="850" Name="P(U) dyn" Value="5.00" Min="0.00" Max="120.00" Unit="s" Var="true" Default="5.00"/>
<GridParameter ID="45" SortID="930" Name="φ PLockIn" Value="10.00" Min="0.00" Max="100.00" Unit="%" Var="true" Default="10.00"/>
<GridParameter ID="46" SortID="940" Name="φ PLockOut" Value="10.00" Min="0.00" Max="100.00" Unit="%" Var="true" Default="10.00"/>
<GridParameter ID="87" SortID="990" Name="φ Limit" Value="0.35" Min="0.14" Max="1.00" Unit="" Var="true" Default="0.35"/>
<GridParameter ID="58" SortID="1000" Name="Q fix" Value="0.00" Min="-1089.00" Max="1089.00" Unit="var" Var="true" Default="0.00"/>
<GridParameter ID="97" SortID="1020" Name="Q PLockIn" Value="0.00" Min="0.00" Max="100.00" Unit="%" Var="true" Default="0.00"/>
<GridParameter ID="98" SortID="1030" Name="Q PLockOut" Value="0.00" Min="0.00" Max="100.00" Unit="%" Var="true" Default="0.00"/>
<GridParameter ID="47" SortID="1040" Name="Q dynamik" Value="3.33" Min="0.00" Max="60.00" Unit="s" Var="true" Default="3.33"/>
<GridParameter ID="38" SortID="1360" Name="R iso Ohm/V" Value="1000.00" Min="0.00" Max="1000.00" Unit="R/V" Var="true" Default="1000.00"/>
<GridParameter ID="48" SortID="1390" Name="RapidShut" Value="1.00" Min="0.00" Max="1.00" Unit="" Var="true" Default="1.00"/>
<GridParameter ID="39" SortID="1420" Name="P UserLimit" Value="2500.00" Min="0.00" Max="2500.00" Unit="W" Var="true" Default="2500.00"/>
<GridParameter ID="82" SortID="1440" Name="PvScanOnOff" Value="1.00" Min="0.00" Max="1.00" Unit="" Var="true" Default="1.00"/>
</GridParameters>
<WebPortals>
<WebPortal Name="PIKO Solar Portal" Protocol="HTTP" URL="www.pikoloop.de" Path="/impsrv" Port="80" Active="true"/>
</WebPortals>
</Device>
</root>
danieldotnl commented 7 months ago

@stephan1827 So I tried to reproduce your scenario. I set up a local web server serving your xml. Then I use your configuration to scrape it. Indeed it doesn't work with parser: xml, but if change that into lxml I don't see the warning from BeautifulSoup. Do you still see that?

stephan1827 commented 7 months ago

Yep, its still there 

2024-04-07 18:28:31.571 WARNING (SyncWorker_2) [py.warnings] /usr/local/lib/python3.12/concurrent/futures/thread.py:58: XMLParsedAsHTMLWarning: It looks like you're parsing an XML document using an HTML parser. If this really is an HTML document (maybe it's XHTML?), you can ignore or filter this warning. If it's XML, you should know that using an XML parser will be more reliable. To parse this document as XML, make sure you have the lxml package installed, and pass the keyword argument `features="xml"` into the BeautifulSoup constructor.