Adding support for a custom chunker. This must be called with a defined interface, chunker(source_data) where source_data is a list of strings, typically representing each line of input in a file. The chunker method must be in a file called custom_chunker.py
Why
Requested by ArguFlow
Verification
Can see that the request to embed and upload an HTML file succeeds:
Can see in the pod logs here that the HTML converts to a string succesfully:
What
embedendpointchunker(source_data)wheresource_datais a list of strings, typically representing each line of input in a file. Thechunkermethod must be in a file calledcustom_chunker.pyWhy
Requested by ArguFlow
Verification
Can see that the request to embed and upload an HTML file succeeds:
Can see in the pod logs here that the HTML converts to a string succesfully:
And can see the custom chunker being called:
Unit tests all still pass