Agents have an option to select what splitter they need to use.
Each splitter should have extra params that the users can define in the frontend per Agent.
Currently, Static Word Count splitter, has a hardcoded limit in the application level (300 words). Users need to be able to define the word count per Agent.
Create a new splitter that splits the text per Chapter (in markdown specified with #, ##, ###, etc). There is an upper limit of each section that is also configurable in the Agent settings. We want to have an average of 500 words per section. The default limit is 600 words. A paragraph in included in the section if the total section length is 600 words or less.
Description
Agents have an option to select what splitter they need to use.
Each splitter should have extra params that the users can define in the frontend per Agent.
Currently, Static Word Count splitter, has a hardcoded limit in the application level (300 words). Users need to be able to define the word count per Agent.
Create a new splitter that splits the text per Chapter (in markdown specified with
#,##,###, etc). There is an upper limit of each section that is also configurable in the Agent settings. We want to have an average of 500 words per section. The default limit is 600 words. A paragraph in included in the section if the total section length is 600 words or less.