ML-405/ML-427 - OntologyElement improvements

Unstructured-IO / unstructured

Open source libraries and APIs to build custom preprocessing pipelines for labeling, training, or production machine learning pipelines.

https://www.unstructured.io/

Apache License 2.0

9.21k stars 764 forks source link

Closed MaksOpp closed 3 weeks ago

MaksOpp commented 3 weeks ago

the "value" attribute from tag will be taken into account and processed as "text" in ontology
the tables will now be parsed without any ids and classes - we have different reasons behind that, for example, embeddings with ids and classes can lose some semantic value. Also, more tokens = more expensive LLM call
cleaned to_html, created to_text for OntologyElement