[feat] PromptFx web crawler improvements

[triathematician]

• [x] Don't crawl duplicate sites with anchors ('#content')
• [x] Clean up extra line breaks between paragraphs
• [x] Use links from "article" div if possible