The ExtractorModel isn't json compatible due to usage of 'bytes'. Pydantic will raise utf8 encoding issues when non unicode chars are present in the byte string.
This PR replaces the json dump logic with model_validate and model_dump. It is done twice to work around some pydantic behavior we observed where sets are only converted to lists on creation of the model.
A test case was added for the model and the verification code to prevent regressions.
The ExtractorModel isn't json compatible due to usage of 'bytes'. Pydantic will raise utf8 encoding issues when non unicode chars are present in the byte string.
This PR replaces the json dump logic with model_validate and model_dump. It is done twice to work around some pydantic behavior we observed where sets are only converted to lists on creation of the model.
A test case was added for the model and the verification code to prevent regressions.