what functionality is ready to try / test out

ChrisCates / CommonCrawler

🕸 A simple way to extract data from Common Crawl

MIT License

33 stars 12 forks source link

Closed RYAN-dot-LOCAL closed 5 years ago

RYAN-dot-LOCAL commented 5 years ago

The readme says coming soon..., are any of the go programs ready for folks to try out today?

ChrisCates commented 5 years ago

Hi @4688-is-great,

You can actually just run go run src/*.go and it should work as intended. Currently, it is stable on Mac OS and Linux.

If you want to modify the parameters for crawling, you'll have to do it manually by updating the config.go file.

ChrisCates commented 5 years ago

Feel free to open issues in regards to feedback. Will be closing this for now.