search

kbr / fritzconnection

Python-Tool to communicate with the AVM Fritz!Box by the TR-064 protocol and the AHA-HTTP-Interface
MIT License
303 stars 59 forks source link

"xml.etree.ElementTree.ParseError: mismatched tag: " after distribution upgrade #163

Closed TauPan closed 1 year ago

TauPan commented 1 year ago

Hi!

I've installed via pip install fritzconnection --user and it worked fine all the time. After a distribution upgrade from openSUSE 15.2 to 15.3 I get the following error:

> FRITZ_USER=Friedel FRITZ_PASSWORD=$(keepassxc-cli show -s -a Password ~NC/secret/Passwords.kdbx 'Fritz!Box 7490') fritzstatus -i 192.168.0.254
Enter password to unlock /home/delgado/Nextcloud/secret/Passwords.kdbx:
Traceback (most recent call last):
  File "/home/delgado/.local/bin/fritzstatus", line 8, in <module>
    sys.exit(main())
  File "/home/delgado/.local/lib/python3.6/site-packages/fritzconnection/cli/fritzstatus.py", line 45, in main
    fs = get_instance(FritzStatus, args)
  File "/home/delgado/.local/lib/python3.6/site-packages/fritzconnection/cli/utils.py", line 36, in get_instance
    use_tls=args.encrypt,
  File "/home/delgado/.local/lib/python3.6/site-packages/fritzconnection/lib/fritzbase.py", line 37, in __init__
    fc = FritzConnection(**kwargs)
  File "/home/delgado/.local/lib/python3.6/site-packages/fritzconnection/core/fritzconnection.py", line 155, in __init__
    self.device_manager.add_description(source)
  File "/home/delgado/.local/lib/python3.6/site-packages/fritzconnection/core/devices.py", line 75, in add_description
    root = get_xml_root(source, timeout=self.timeout, session=self.session)
  File "/home/delgado/.local/lib/python3.6/site-packages/fritzconnection/core/utils.py", line 79, in get_xml_root
    return etree.fromstring(source)
  File "/usr/lib64/python3.6/xml/etree/ElementTree.py", line 1314, in XML
    parser.feed(text)
xml.etree.ElementTree.ParseError: mismatched tag: line 117, column 2

lxml is version 4.7.1 now. According to https://software.opensuse.org/package/python-lxml it used to be 4.4.2 before. My Fritz!Box 7490 hasn't received an update or a configuration change during that period.

As always lxml gives precious little information. I've tried to run in pdb but apparently the exception is caught and I can't figure out right now how to halt in the correct frame here in order to figure out which tag might be giving problems and what the xml payload might look like.

As a side note in a project of mine I also had trouble with more recent versions of lxml breaking things, but I didn't have time to debug those yet either. (But it's a different error.)

Version of libxml2 is

libxml2-2-2.9.7-150000.3.46.1.x86_64

Anything else you might need, please just ask!

kbr commented 1 year ago

The library can log the data transferred with the router. It's a bit hidden in the docs: https://fritzconnection.readthedocs.io/en/1.9.1/sources/api.html#module-fritzconnection.core.logger

However, since version 1.0 fritzconnection does not require lxml anymore. Did you do the installation in the system-python? Doing this can potentially break things between python-applications and also on system updates. Best is to keep the system-python as is and use virtual environments for everything else.

TauPan commented 1 year ago

Sorry, I didn't read the backtrace properly. It's not lxml but xml.etree.ElementTree, in this case from python 3.6 (system).

Same thing happens with python 3.6.8 from pyenv in a fresh virtualenv:

(fritzconnection) delgado@bluki:~AS> export FRITZ_USER=Friedel; export FRITZ_PASSWORD=$(keepassxc-cli show -s -a Password ~NC/secret/Passwords.kdbx 'Fritz!Box 7490'); fritzstatus -i 192.168.0.254                                        git:(master)  16:19
Enter password to unlock /home/delgado/Nextcloud/secret/Passwords.kdbx:
Traceback (most recent call last):
  File "/home/delgado/.virtualenvs/fritzconnection/bin/fritzstatus", line 8, in <module>
    sys.exit(main())
  File "/home/delgado/.virtualenvs/fritzconnection/lib/python3.6/site-packages/fritzconnection/cli/fritzstatus.py", line 45, in main
    fs = get_instance(FritzStatus, args)
  File "/home/delgado/.virtualenvs/fritzconnection/lib/python3.6/site-packages/fritzconnection/cli/utils.py", line 36, in get_instance
    use_tls=args.encrypt,
  File "/home/delgado/.virtualenvs/fritzconnection/lib/python3.6/site-packages/fritzconnection/lib/fritzbase.py", line 37, in __init__
    fc = FritzConnection(**kwargs)
  File "/home/delgado/.virtualenvs/fritzconnection/lib/python3.6/site-packages/fritzconnection/core/fritzconnection.py", line 155, in __init__
    self.device_manager.add_description(source)
  File "/home/delgado/.virtualenvs/fritzconnection/lib/python3.6/site-packages/fritzconnection/core/devices.py", line 75, in add_description
    root = get_xml_root(source, timeout=self.timeout, session=self.session)
  File "/home/delgado/.virtualenvs/fritzconnection/lib/python3.6/site-packages/fritzconnection/core/utils.py", line 79, in get_xml_root
    return etree.fromstring(source)
  File "/home/delgado/git/pyenv/versions/3.6.8/lib/python3.6/xml/etree/ElementTree.py", line 1314, in XML
    parser.feed(text)
xml.etree.ElementTree.ParseError: mismatched tag: line 117, column 2

And with the system python3 which is 3.6.15:

(fritzconnection) delgado@bluki:~AS> export FRITZ_USER=Friedel; export FRITZ_PASSWORD=$(keepassxc-cli show -s -a Password ~NC/secret/Passwords.kdbx 'Fritz!Box 7490'); fritzstatus -i 192.168.0.254                                        git:(master)  16:21
Enter password to unlock /home/delgado/Nextcloud/secret/Passwords.kdbx:
Traceback (most recent call last):
  File "/home/delgado/.virtualenvs/fritzconnection/bin/fritzstatus", line 8, in <module>
    sys.exit(main())
  File "/home/delgado/.virtualenvs/fritzconnection/lib/python3.6/site-packages/fritzconnection/cli/fritzstatus.py", line 45, in main
    fs = get_instance(FritzStatus, args)
  File "/home/delgado/.virtualenvs/fritzconnection/lib/python3.6/site-packages/fritzconnection/cli/utils.py", line 36, in get_instance
    use_tls=args.encrypt,
  File "/home/delgado/.virtualenvs/fritzconnection/lib/python3.6/site-packages/fritzconnection/lib/fritzbase.py", line 37, in __init__
    fc = FritzConnection(**kwargs)
  File "/home/delgado/.virtualenvs/fritzconnection/lib/python3.6/site-packages/fritzconnection/core/fritzconnection.py", line 155, in __init__
    self.device_manager.add_description(source)
  File "/home/delgado/.virtualenvs/fritzconnection/lib/python3.6/site-packages/fritzconnection/core/devices.py", line 75, in add_description
    root = get_xml_root(source, timeout=self.timeout, session=self.session)
  File "/home/delgado/.virtualenvs/fritzconnection/lib/python3.6/site-packages/fritzconnection/core/utils.py", line 79, in get_xml_root
    return etree.fromstring(source)
  File "/usr/lib64/python3.6/xml/etree/ElementTree.py", line 1314, in XML
    parser.feed(text)
xml.etree.ElementTree.ParseError: mismatched tag: line 117, column 2
(fritzconnection) delgado@bluki:~AS[1]> python3                                                                                                                                                                                            git:(master)  16:21
Python 3.6.15 (default, Sep 23 2021, 15:41:43) [GCC] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

And the same backtrace with a recently installed 3.10.5 from pyenv.

I guess I'll have to look at the xml payload via the logger module. I'll try that some time later. Thanks for the hint!

TauPan commented 1 year ago

Argh, my systemadministrator advised me to set a system wide HTTP_PROXY variable before the upgrade and I forgot to remove it, so fritzstatus was trying to access my fritzbox over the company proxy. :man_facepalming:

(But I indeed figured that out by activating the logging, thanks!)

Sorry for the noise.