ChrisCates / CommonCrawler

🕸 A simple way to extract data from Common Crawl
MIT License
33 stars 12 forks source link

Full usage as a library #13

Open ChrisCates opened 5 years ago

ChrisCates commented 5 years ago

Summary

This repository should be accessible via go get and can be included easily into anyone else's project.

Requirements

Must be able to run:

go get https://github.com/ChrisCates/CommonCrawler

Must be able to access the library as:

import(
  cc "github.com/ChrisCates/CommonCrawler"
)

func main() {
  cc.scan()
  cc.download()
  cc.extract()
  // And so forth
}

The library must be stable and also have a demo example in the repository. Demoing the library should be part of the Travis CI configuration.

Payment

1 ETH will be paid once meeting the requirements.