vbauerster / mpb

multi progress bar for Go cli applications
The Unlicense
2.29k stars 123 forks source link

Performance issues #99

Closed xtreding closed 2 years ago

xtreding commented 3 years ago

Hi guys! First of all, the project is amazing! Thank you for your efforts!

My problem

I noticed that if I create more than a hundreds progress bars, then my service consumes about 99% CPU.

Conditions

I use multi progress bars in CLI tool that uploads files into API. Usually my use-case is to upload thousands files (~30,000). I create one main progress bar that shows whole operation progress N/30,000, N+1/30,000 and I create one progress bar per file with onComplete option.

As the result I have the following picture in the terminal:

image

Do you have any solution - how can I keep records with "✓ done" (I can't use BarRemoveOnComplete()) and don't face performance issues? Thank you!

Code example

import(
au "github.com/logrusorgru/aurora"
    "github.com/vbauerster/mpb/v7"
    "github.com/vbauerster/mpb/v7/decor"
    "github.com/hashicorp/go-uuid"
)

func main() {
    mp := mpb.New()

    gb := mp.Add(30000,
        mpb.NewBarFiller(mpb.BarStyle().Lbound("╢").Filler(au.Index(93, "█").String()).Tip("").Padding(au.Index(99, "░").String()).Rbound("╟")),
        mpb.PrependDecorators(
            decor.Name("replay"),
            decor.Percentage(decor.WCSyncSpace),
        ),
        mpb.AppendDecorators(
            decor.OnComplete(
                decor.CountersNoUnit("%d / %d files", decor.WCSyncWidth), au.Green("✓ done").String(),
            ),
        ),
        mpb.BarPriority(9999999),
    )

    for i:=0;i<30000;i++{
        u, _ := uuid.GenerateUUID()
        fileBar := mp.Add(1,
            mpb.NewBarFiller(mpb.BarStyle().Lbound("╢").Filler(au.Index(99, "█").String()).Tip("").Padding(au.Index(104, "░").String()).Rbound("╟")),
            mpb.BarFillerClearOnComplete(),
            mpb.PrependDecorators(
                decor.Name(u + u),
                decor.Percentage(decor.WCSyncSpace),
            ),
            mpb.AppendDecorators(
                decor.OnComplete(
                    decor.CountersKiloByte("%d / %d", decor.WCSyncWidth), au.Green("✓ done").String(),
                ),
            ),
        )
        time.Sleep(time.Millisecond * 100)
        fileBar.SetCurrent(1)
        gb.SetCurrent(int64(i + 1))
    }

    time.Sleep(5 * time.Minute)
}
vbauerster commented 3 years ago

Make sure you use latest v7.1.3 and try without aurora package. If you start up 30000 bars at the same time without any semaphore, then high cpu usage is expected and may be not due to mpb. I can't reproduce on my machine because my screen fits about 70 lines, but even with 70 simultaneous bars there is no performance issues.