aws-samples / bedrock-claude-chat

AWS-native chatbot using Bedrock + Claude (+Mistral)
MIT No Attribution
908 stars 324 forks source link

feat: Add Web Crawler for Knowledge Bases #600

Closed fsatsuki closed 1 day ago

fsatsuki commented 3 days ago

Issue #, if available:]

557

Description of changes: Add Web Crawler for Knowledge Bases. We can set 10 URLs for websites to be crawled. You can also set URL inclusion/exclusion patterns.

I'm reusing v1 sourceUrls that aren't used in v2. Since Youtube is not supported, the description about Youtube has been removed.

Screenshot 2024-11-19 at 10 41 55

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.