Brave Default Setting Improvements

[Shrinks99]

Context

Some of Brave's settings aren't good defaults for crawling.

• Embedded LinkedIn posts are blocked by default to preserve privacy — probably with good intention, but it also means they won't be captured by the crawler!
• AMP pages are redirected automatically — unsure if this is good or bad, seems like possible added hassle when figuring out scoping?
• Tracker and ad blocking is set to Standard, unsure what this does
• Cross site cookies are blocked — likely fine?
• Widevine is off by default — likely not an issue, crawling DRM content doesn't work.
• IPFS is set to "Ask" and requires user input, won't work when crawling
• Brave's analytics are on, would probably be good to turn these off? We're not real users :P
• Brave's shields (default adblock and tracking protection) are on by default

User Story

As a user, I'd like my crawler's default settings to offer the best chances of success!

[tw4l]

Thank you for doing this investigation! I agree that it'd be wise to tweak the default profile to fit user expectations a bit better.

We could also potentially have a selection of "base profiles" that users could extend from or ignore as they wish, e.g.:

1. All Shields Disabled
2. Webrecorder default
3. Aggressive Ad + Cookie Blocking