Closed scanny closed 2 months ago
What are the four decorators this is replacing? I'm only seeing three that are used on the partitioners:
process_metadata
.metadata.regex-metadata
when regex_metadata
keyword-argument is provided.unique_element_ids
argument is provided and True.add_metadata_with_filetype
add_chunking_strategy
chunking_strategy
argument is present in the partitioner call and it names an available chunking strategy.Also, I'm not seeing the chunk addressed in this decorator. Is that coming in a later PR?
Summary In preparation for pluggable auto-partitioners, add a new metadata decorator to replace the four existing ones.
Additional Context "Global" metadata items, those applied to all element on all partitioners, are applied using a decorator.
Currently there are four decorators where there only needs to be one. Consolidate those into a single metadata decorator. One or two additional behaviors of the new decorator will allow us to remove decorators from delegating partitioners which is a prerequisite for pluggable auto-partitioners.