search

si9int / cc.py

Extracting URLs of a specific target based on the results of "commoncrawl.org"
MIT License
268 stars 47 forks source link

I assume I need to update the values in index.txt? #11

Open aaronsql2019 opened 2 years ago

aaronsql2019 commented 2 years ago

I was hoping to use this project to look at some newer data. I assume I should just add the name of the indexes in the file 'index.txt'?

user9825 commented 8 months ago

When you use the cc.py with -u flag, it updates the index file by itself from => https://index.commoncrawl.org/collinfo.json

aaronsql2019 commented 6 months ago

Thanks so much.