search

audeering / audb

Manage audio and video databases
https://audeering.github.io/audb/
Other
23 stars 1 forks source link

Progress bar estimated remaining time too erratic #442

Closed hagenw closed 2 months ago

hagenw commented 2 months ago

This might be an unwanted side effect of https://github.com/audeering/audb/pull/440.

When publishing a database with lots of files, using multiple workers, the estimation of the remaining time is very erratic:

progress-bar

We might be able to fix this by using smoothing=0 in tqdm.tqdm(), or maybe adjusting its miniters argument.

hagenw commented 2 months ago

I checked, and it seems that the progress bar behaved very similar before #440. The time estimation can in general be very erratic.

hagenw commented 2 months ago

Using tqdm.tqdm(..., smoothing=0) does indeed improve the behavior for audb<1.8 and audb==1.8. In addition, it works when using a single worker or multiple workers.

The only downside is, that we cannot pass arguments to tqdm.tqdm() from audeer.run_tasks(), which means we need to update audeer first in order to be able to make the changes in audb.