add CI script to check for rewards requests before rewards opt-in

[diracdeltas]

per https://bravesoftware.slack.com/archives/CB0KCRCQN/p1613880512000800, we need better automated checks that rewards domains are not pinged before rewards is opted into.

subtasks:

1. remove any domains from https://github.com/brave/brave-core/blob/9f2edc372ea22440dc382bfca758d3536dfbe022/build/commands/lib/whitelistedUrlPrefixes.js which are rewards related
2. add a browsertest similar to the network-audit script which starts the browser, navigates to twitter/github/youtube/reddit/etc. pages, and then fails if any rewards domains are found in the network log. (see https://github.com/brave/brave-core/blob/master/build/commands/lib/start.js#L130 for how to do this)

[diracdeltas]

we also should be doing the network-audit check on android builds. cc @SergeyZhukovsky @jumde

[BrendanEich]

Just for transparency as this is a public github issue, the slack link above is to an item from me that links to

https://www.youtube.com/watch?v=krB0enBeSiE&lc=UgzKw3iT6CR_wXjjDEh4AaABAg.9JuBRgCp30M9K-t98EyBnh

fourth item, to get this fixed.

[kjozwiak]

@diracdeltas believe we can close this off as it looks like https://github.com/brave/brave-core/pull/8035 has been merged?

[diracdeltas]

https://github.com/brave/brave-core/pull/8035 is mostly a cleanup task; the main part of this issue is to add a new CI script which checks more comprehensively (ex: visiting twitter.com with ads disabled)

[KoushikBaagh]

Hey, @diracdeltas

The new files are : a) network_audit_browsertest.cc : https://github.com/brave/brave-core/blob/master/browser/net/brave_network_audit_browsertest.cc b) network_audit_allowed_lists.h : https://github.com/brave/brave-core/blob/master/browser/net/brave_network_audit_allowed_lists.h

Are these information correct ?
Am I missing anything ? Is there anything that you all would like to add as "guidance/advice" , so that it might help me while working on it ?

CC:- @BrendanEich @kjozwiak