Get a list of every Ask Media Group website

[Jack5079]

To build up the list of SEO microsites used in #27

          > We should find a way to scrape all System1 and Ask Media Group domains

All Ask Media Group domains have /terms with a <div class="class="terms-of-service-title">Ask Media Group, LLC Terms of Service</div>

Originally posted by @Jack5079 in https://github.com/mollersuite/tangent/issues/27#issuecomment-1913836752

[Jack5079]

Those terms have <meta name="robots" content="noindex"> on them so it will be harder

[Jack5079]

Three kinds of Ask Media Group properties:

1. Yahoo wrappers. These are all noindex but you might be redirected to them by arbitrage ad campaigns or malware. They have /terms, and tend to be on domains that used to be independent search engines.
   • Example: informationvine.com
2. Blogs. They run WordPress. Indexed. They also have a Yahoo wrapper built in. They have /terms.
   • Example: ask.com, reference.com
3. template:iac2. Indexed. They do not have /terms. Custom CMS?
   • Example: explore.informationvine.com

[Jack5079]

https://github.com/search?q=repo%3Acitp%2Fprivacy-policy-historical+%22ask+media+group%22&type=code

[Jack5079]

So far

ask.com
bloglines.com
candofinance.com
consumersearch.com
directhit.com
finecomb.com
govtsearches.com
homeandgardenideas.com
idealhomegarden.com
informationvine.com
internetcorkboard.com
investopedia.com
kensaq.com
life123.com
pageset.com
pronto.com
prontohome.com
prontostyle.com
prontotech.com
reference.com
shop411.com
shopping.net
sidewalk.com
simpli.com
smarter.com
smarterschooling.com
symptomfind.com

[Jack5079]

https://docs.google.com/spreadsheets/d/1puymaStovHq7jBTJH15M6jww0VPy27FRvslaHW7DVzs/edit?usp=sharing has an Ask Media Group section