vbauerster / mpb

multi progress bar for Go cli applications
The Unlicense
2.29k stars 123 forks source link

[QUESTION] Bar doesn't display right info. #121

Open AYehia0 opened 1 year ago

AYehia0 commented 1 year ago

First, I would like to thank you for this cool project. I have some troubles making it work properly, I would really appreciate your help :smile:

I am downloading a playlist of tracks, and I am trying to display multiple bars for each track but I am having some issues since the download size isn't known as I am streaming a m3u8 track and downloading segments into one file as follows :


func Download(track DownloadTrack, dlpath string, prog *mpb.Progress) string {
         // validation ...
        // ...
    // check if the track is hls
    if track.Quality != "low" {
                // downloading the m3u8 file to extract the segments
        resp, err := http.Get(track.Url)
        if err != nil {
            return ""
        }
        defer resp.Body.Close()

        segments := getSegments(resp.Body)
        DownloadM3u8(path, prog, segments)

        return path
    }
    // more code
    // ...
}

and I am using concurrency to download multiple tracks :

var wg sync.WaitGroup
plDownloadTracks := getPlaylistDownloadTracks(soundData, clientId)
p := mpb.New(mpb.WithWaitGroup(&wg),
    mpb.WithWidth(64),
    mpb.WithRefreshRate(180*time.Millisecond),
)

for _, dlT := range plDownloadTracks {
    wg.Add(1)
    go func(dlT []soundcloud.DownloadTrack) {
        defer wg.Done()

        t := getTrack(dlT, true)
        fp := soundcloud.Download(t, downloadPath, p)

        // silent indication of already existing files
        if fp == "" {
            return
        }
        soundcloud.AddMetadata(t, fp)
    }(dlT)
}
wg.Wait()

fmt.Printf("\n%s Playlist saved to : %s\n", theme.Green("[-]"), theme.Magenta(downloadPath))
return

what's interesting is the DownloadM3u8, I create a progressbar and start downloading segments:

// using the goroutine to download each segment concurrently and wait till all finished
func DownloadM3u8(filepath string, prog *mpb.Progress, segments []string) error {

    file, _ := os.OpenFile(filepath, os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0644)
    var wg sync.WaitGroup

        // I create a progressbar with 0 totalSize
    dlbar := prog.AddBar(0,
        mpb.PrependDecorators(
            decor.CountersKibiByte("% .2f / % .2f"),
        ),
        mpb.AppendDecorators(
            decor.EwmaETA(decor.ET_STYLE_GO, 90),
            decor.Name(" -- "),
            decor.EwmaSpeed(decor.UnitKiB, "% .2f", 60),
        ),
    )
    for _, segment := range segments {
        wg.Add(1)
        downloadSeg(&wg, segment, file, dlbar)
    }
    return nil
}

in the downloadSeg function I am copying the data to the file and incrementing the bar :

func downloadSeg(wg *sync.WaitGroup, segmentURI string, file *os.File, dlbar *mpb.Bar) {
    resp, err := http.Get(segmentURI)
    if err != nil {
        return
    }
    defer resp.Body.Close()
    // reader := dlbar.ProxyReader(resp.Body)
    // defer reader.Close()
    _, err = io.Copy(file, reader)
    dlbar.IncrInt64(resp.ContentLength)
    dlbar.SetTotal(dlbar.Current()+resp.ContentLength, false)

    if err != nil {
        return
    }
}

the problem with dlbar.SetTotal() is doesn't reach 100% also when using ProxyReader it displays wrong final files sizes. Here the sizes are right but no speed indication.

mpb.gif

when swapping the 2 lines IncrInt64 with SetTotal it reaches 100% in no time. also adding prog.Wait() hangs the program after finishing.

sorry If I am making it complicated, I am a beginner in go.