search

commoncrawl / cc-index-server

Common Crawl Index Server
http://index.commoncrawl.org/
65 stars 18 forks source link

fix: Removed no-sign-request option in install-collections script #10

Closed ChillarAnand closed 2 years ago

ChillarAnand commented 2 years ago

AWS has removed public access for S3 files. Updated read me & script for the same.

Refer: https://commoncrawl.org/2022/03/introducing-cloudfront-access-to-common-crawl-data/

ChillarAnand commented 2 years ago

Hi, @ChillarAnand! Thanks! Would you mind to change/extend the description. Otherwise let me know and I'll update it. Also to clarify that there are two ways to access CC data including the CDX index.

Updated readme page as well as description. Free free to update further if needed.

sebastian-nagel commented 2 years ago

Thanks, @ChillarAnand!