DataFog / datafog-python

Open source PII detection and anonymization tool: easy-to-use, configurable, and extensible
https://www.datafog.ai
MIT License
9 stars 3 forks source link

python text pipeline not operational #54

Open heijligers opened 1 week ago

heijligers commented 1 week ago

Looks like the text_pipeline is not operational. i'm trying to use scan and replace, but the replace expects annotations of type AnnotationResults, But it's getting normal strings or empty lists from the annotator

Error in run_text_pipeline_sync: 'str' object has no attribute 'start'
Traceback (most recent call last):
  File "/Users/datascience/PycharmProjects/FF/Topologix/src/SQLModel_ff_db.py", line 2705, in <module>
    replaced_text = client.run_text_pipeline_sync([content])[0]
  File "/Users/datascience/PycharmProjects/FF/Topologix/.venv/lib/python3.10/site-packages/datafog/main.py", line 200, in run_text_pipeline_sync
    return [
  File "/Users/datascience/PycharmProjects/FF/Topologix/.venv/lib/python3.10/site-packages/datafog/main.py", line 201, in <listcomp>
    self.anonymizer.anonymize(text, annotations).anonymized_text
  File "/Users/datascience/PycharmProjects/FF/Topologix/.venv/lib/python3.10/site-packages/datafog/models/anonymizer.py", line 56, in anonymize
    return self.replace_pii(text, annotations)
  File "/Users/datascience/PycharmProjects/FF/Topologix/.venv/lib/python3.10/site-packages/datafog/models/anonymizer.py", line 67, in replace_pii
    for annotation in sorted(annotations, key=lambda x: x.start, reverse=True):
  File "/Users/datascience/PycharmProjects/FF/Topologix/.venv/lib/python3.10/site-packages/datafog/models/anonymizer.py", line 67, in <lambda>
    for annotation in sorted(annotations, key=lambda x: x.start, reverse=True):
AttributeError: 'str' object has no attribute 'start'
sidmohan0 commented 1 week ago

Hey @heijligers - thanks for flagging - I'll check this out myself later today and follow up

sidmohan0 commented 1 week ago

do you mind sharing the datafog python version that you're using @heijligers?