xinsnake / go-http-digest-auth-client

Golang Http Digest Authentication Client
BSD 2-Clause "Simplified" License
41 stars 68 forks source link

net/http: request canceled (Client.Timeout exceeded while awaiting headers) #9

Closed vipin733 closed 6 years ago

vipin733 commented 6 years ago

I'm trying to call an external api in loop using go concurrency at one time nearly 16k but I'm getting error

net/http: request canceled (Client.Timeout exceeded while awaiting headers)

here is my impletenrtin

` var wg sync.WaitGroup wg.Add(16367)

// fmt.Println("Running for loop…")

for results.Next() {
    var tag StationData
    // for each row, scan the result into our tag composite object
    err = results.Scan(&tag.ID, &tag.Station)
    if err != nil {
        panic(err.Error()) // proper error handling instead of panic in your app
    }
    id := tag.ID
    stationname := tag.Station
    go func(id int) {
        defer wg.Done()
        time.Sleep(time.Millisecond * 500)
        t := dac.NewTransport("username", "password")
        req, err := http.NewRequest("GET", "urlsomttthh", nil)

        if err != nil {
            log.Fatalln(err)
        }

        response, err := t.RoundTrip(req)
        if err != nil {
            fmt.Printf("somthing missing from data %s\n", err)
        } else {
            data, _ := ioutil.ReadAll(response.Body)
            fmt.Println(string(data))
        }
    }(id)

    // datasprin := "somthing" + tag.Station
    // and then print out the tag's Name attribute
    // log.Printf(datasprin)

}

wg.Wait()
// fmt.Println("Finished for loop")
fmt.Printf("total time %s", time.Now().Sub(start))

xinsnake commented 6 years ago

@vipin733 Sorry for the delayed response. Can you please confirm whether this issue still require attention? If yes please re-open. Thanks!

deveshmishra34 commented 3 years ago

Hi i'm also facing the same error and my use case is same, I'm trying to upload bulk files concurrently ...