Open michielbdejong opened 6 years ago
For instance when visiting cnet.com you agree to the cbsinteractive.com privacy policy. Should we crawl that as rules/cnet.com.xml or as rules/cbsinteractive.com.xml? I opted for the first now, but this issue deserves more attention.
also for instance apple-icloud is a separate services, but on tosback its terms are crawled as part of apple.com
For instance when visiting cnet.com you agree to the cbsinteractive.com privacy policy. Should we crawl that as rules/cnet.com.xml or as rules/cbsinteractive.com.xml? I opted for the first now, but this issue deserves more attention.