Thanks to customer feedaback I am jumping on this big feature.
Any source will be 100% driven by the Prompt the user put.
So Source Email, Web, Search etc all gets passed to an LLM to do something with the text.
For example you want to check emails about xyz only, and if it is an email about that the prompt will process that text and turn it into a document in your collection. Or scrape this site for event data, the LLM will then use the HTML to just get you the event data.
There is a lot to do here:
[ ] All Sources Converted to this pattern (false or not, array or not)
[x] WebPageSource
[x] WebSearchSource
[x] WebhookSource
[x] SiteMapSource
[x] JsonSource
[x] GoogleSheetSource
[x] FeedSource
[x] EmailSource
[x] EmailBoxSource
[x] All Sources allow for a way to NOT delete emails or NOT take in data kinda of like a false return to not save as document
[x] UI will surface this easily for the user so they are empowered to do this work
[x] Prompts can product Array
[x] SourceTask to prevent repeat on emails without deleting or marking emails.
[x] Chat now shows Sources and Messages to show a log of activity to the user
[x] Chunking is a generic job since all of this is text after this point
Thanks to customer feedaback I am jumping on this big feature.
Any source will be 100% driven by the Prompt the user put.
So Source Email, Web, Search etc all gets passed to an LLM to do something with the text.
For example you want to check emails about xyz only, and if it is an email about that the prompt will process that text and turn it into a document in your collection. Or scrape this site for event data, the LLM will then use the HTML to just get you the event data.
There is a lot to do here: