Open pandeyganesh-dc opened 1 month ago
For some connection, If I choose SplitByHtmlHeaderSettings and later decide to edit configuration and choose SplitByCharacterSettings, other options remain in the catalog like headers_to_split_on and doesn't get options like seperator
SplitByHtmlHeaderSettings
SplitByCharacterSettings
headers_to_split_on
seperator
Sample catalog dict :
"catalog": { "document_streams": [ { "name": "html", "json_schema": null, "namespace": "idk4", "read_sync_mode": "FULL_REFRESH", "write_sync_mode": "APPEND", "cursor_field": "", "advanced": { "splitter_settings": { "splitter_settings": "SPLIT_BY_CHARACTER", "headers_to_split_on": [ "h2", "h3" ] } }, "usr_file_path": "dummy", "obj_file_path": "kaushiki.html" } ] },
So, I have to make a new connection.
Local File System
OpenAI
qdrant
html
network
It would reproduce the error.
No response
Is there an existing issue for this?
Description
For some connection, If I choose
SplitByHtmlHeaderSettings
and later decide to edit configuration and chooseSplitByCharacterSettings
, other options remain in the catalog likeheaders_to_split_on
and doesn't get options likeseperator
Sample catalog dict :
So, I have to make a new connection.
Steps to Reproduce
Local File System
->OpenAI
->qdrant
in my casehtml
stream and configure it to useSplitByHtmlHeaderSettings
SplitByCharacterSettings
this timenetwork
tab and run the connectionIt would reproduce the error.
Screenshots
Environment
Additional Context
No response