Describe the bug
The CCT command measure-table-structure-accuracy-command doesn't drop the extra index when it doesn't find a table to process (i.e. the documents have the wrong format).
File "/Users/mallori/unstructured/unstructured/ingest/evaluate.py", line 276, in <module>
main()
File "/Users/mallori/opt/anaconda3/lib/python3.9/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
File "/Users/mallori/opt/anaconda3/lib/python3.9/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/Users/mallori/opt/anaconda3/lib/python3.9/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/mallori/opt/anaconda3/lib/python3.9/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/mallori/opt/anaconda3/lib/python3.9/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/Users/mallori/unstructured/unstructured/ingest/evaluate.py", line 236, in measure_table_structure_accuracy_command
return measure_table_structure_accuracy(
File "/Users/mallori/unstructured/unstructured/metrics/evaluate.py", line 375, in measure_table_structure_accuracy
agg_df.columns = agg_headers
File "/Users/mallori/opt/anaconda3/lib/python3.9/site-packages/pandas/core/generic.py", line 5915, in __setattr__
return object.__setattr__(self, name, value)
File "pandas/_libs/properties.pyx", line 69, in pandas._libs.properties.AxisProperty.__set__
File "/Users/mallori/opt/anaconda3/lib/python3.9/site-packages/pandas/core/generic.py", line 823, in _set_axis
self._mgr.set_axis(axis, labels)
File "/Users/mallori/opt/anaconda3/lib/python3.9/site-packages/pandas/core/internals/managers.py", line 230, in set_axis
self._validate_set_axis(axis, new_labels)
File "/Users/mallori/opt/anaconda3/lib/python3.9/site-packages/pandas/core/internals/base.py", line 70, in _validate_set_axis
raise ValueError(
ValueError: Length mismatch: Expected axis has 6 elements, new values have 5 elements
Describe the bug The CCT command
measure-table-structure-accuracy-command
doesn't drop the extra index when it doesn't find a table to process (i.e. the documents have the wrong format).To Reproduce
Expected behavior
Screenshots Error
Environment Info
Additional context Add any other context about the problem here.