tefra / xsdata

Naive XML & JSON Bindings for python
https://xsdata.readthedocs.io
MIT License
310 stars 56 forks source link

Simultaneous parsing sometimes causes "ConverterWarning: Failed to convert value" #1041

Closed DareDevilDenis closed 2 months ago

DareDevilDenis commented 2 months ago

Using:

With the following script that repeatedly performs the same parse operation on different threads:

import threading
from pathlib import Path
from xsdata.formats.dataclass.parsers import XmlParser
from xsdata.formats.dataclass.parsers.handlers import XmlEventHandler
from my_schema import LogRecord

input_xml_path = Path(__file__).parent / "input.xml"
parser = XmlParser(handler=XmlEventHandler)

with open(input_xml_path, "r", encoding="utf-8-sig") as file:
    input_xml=file.read()

print("Started...")
for i in range(2000):
    print(f"Iter {i}")
    threading.Thread(target=parser.from_string, args=(input_xml, LogRecord)).start()

print("Finished")

After a while we get the following warning:

Iter 352
Iter 353
Iter 354
C:\Program Files\Python312\Lib\site-packages\xsdata\formats\converter.py:108: ConverterWarning: Failed to convert value `0002 | 0052 | 5 | 0 mu1` to one of (<class 'int'>,)
  warnings.warn(
Iter 355
Iter 356
Iter 357

Here are the files: ConverterWarning_issue.zip

tefra commented 2 months ago

On union fields, the parser attempts to parse every candidate class and the best candidate wins, but any warning are supposed to be suppressed.

tefra commented 2 months ago

Ahh excellent, catch_warnings is not threadsafe 🤦 https://docs.python.org/3.7/library/warnings.html#warnings.catch_warnings

tefra commented 2 months ago

Thanks for reporting @DareDevilDenis, excellent example!

The fix is on main!

DareDevilDenis commented 2 months ago

And thank you @tefra for the quick fix! 👍