gocolly / colly

Elegant Scraper and Crawler Framework for Golang
https://go-colly.org/
Apache License 2.0
23.2k stars 1.76k forks source link

Add iteration api for context #77

Closed rongyi closed 6 years ago

rongyi commented 6 years ago

When I use colly, I have a case to to iterate context elements when I put something in it with multiple OnHTML callback on different html elements. This is the simple function I wrote.

// ForEach iterate context
func (c *Context) ForEach(fn func(k string, v interface{}) interface{}) []interface{} {
    c.lock.RLock()
    defer c.lock.RUnlock()

    ret := make([]interface{}, 0, len(c.contextMap))
    for k, v := range c.contextMap {
        cur := fn(k, v)
        ret = append(ret, cur)
    }

    return ret
}

Hope this can help someone when they also need to iterate context.

asciimoo commented 6 years ago

@rongyi great, this is a useful feature. Could you please open a PR with the above changes?

rongyi commented 6 years ago

@asciimoo send you a PR :)

asciimoo commented 6 years ago

implemented in #79