search

versioneye / crawl_r

VersionEye crawlers implemented in Ruby.
https://www.versioneye.com
Other
10 stars 7 forks source link

#53 - add godependency crawler for Godeps files on Github; #54

Open timgluz opened 7 years ago

timgluz commented 7 years ago

Hi,

this crawler exposes method crawl_product(gopkg_id, commit_sha, branch, tag), which can fetch list of project dependencies from Github. It's useful for pulling additional dependency and version information for Go packages crawled by GosearchCrawler.

The crawler is looking for static files for the supported Go project managers: Dep, Glide, Godeps and Govendor

The function returns list of Projectdependency or nil if no file was found

Usage:

# pull dependency file by commit sha
GodepsGithubCrawler.crawl_product('k8s.io/kubernetes', 'b06a9b8554b72145daa844a32d08bc9a09424788')

# pull dependency file by commit branch
GodepsGithubCrawler.crawl_product('k8s.io/kubernetes', 'release-1.7')

#pull dependency file by commit tag
GodepsGithubCrawler.crawl_product('k8s.io/kubernetes', 'v1.6.3')
timgluz commented 7 years ago

It's ready for review and for usage.

I'm going to add support for other parsers and then it could pull data from other files too.

timgluz commented 7 years ago

OK, will need to update this branch.

and i need to review how does it save submodules.

timgluz commented 7 years ago

Going to review it today;