Add URL based allowlist filter on sources

[dspruell-i01]

Feature request: implement capability to filter ingested pages from sources like blogs/sitemaps/etc. based on URL patterns.

Use case:

• Suppose an RSS source is set up for ingestion of content for a given blog.
• Suppose that all published content from this source is desired for ingestion, except for publications matching a certain URL pattern.
• User wants to add a filter to the source to ignore ingestion for linked pages matching a certain page URL pattern.
• Example:
  • Ingest from source:
    • https://blog.talosintelligence.com/
  • Ignore ingestion for URL pattern:
    • /^https:\/\/blog\.talosintelligence\.com\/threat-roundup-/
  • Resulting ignored/skipped page examples:
    • https://blog.talosintelligence.com/threat-roundup-0407-0414/
    • https://blog.talosintelligence.com/threat-roundup-0421-0428-2/

The result should be that all content from the source will be ingested, except resources matching the specified URL pattern. This should have precedence over other configuration options that might specify keywords to match for ingestion, for example.