Precision Errors with Floating-point total/update

[colinww]

• [x] I have marked all applicable categories:
  • [ ] exception-raising bug
  • [x] visual output bug
  • [ ] documentation request (i.e. "X is missing from the documentation." If instead I want to ask "how to use X?" I understand StackOverflow#tqdm is more appropriate)
  • [ ] new feature request
• [x] I have visited the source website, and in particular read the known issues
• [x] I have searched through the issue tracker for duplicates
• [x] I have mentioned version numbers, operating system and environment, where applicable:

  print(tqdm.__version__, sys.version, sys.platform)
  4.46.1 3.7.7 (default, Mar 26 2020, 10:32:53) 
  [Clang 4.0.1 (tags/RELEASE_401/final)] darwin

First of all, I really love this module, thank you for developing it! I am using it to show the progress of a simulation. The aspects of this simulation are:

• Track an elapsed interval of time (floating point)
• The progress jump in between updates is variable length

The simulator internally computes the timestep using a custom number format that is more precise than double. It can report the current simulation time approximately as a double. So the original code does something like this:

del_time = sim_tstop - sim_tstart
bar = tqdm(total=del_time, ncols=79)
while SimState.SIM_RUN == self.csim.cstate:
    bar.update(self.csim.tedge - self.csim.pedge)
    # Actual simulator code below
    # ...
    # Update previous timestep
    self.csim.pedge = self.csim.tedge
    # Compute new timestep
    # ...

Thus, I am updating the progress bar using the approximate (as double-precision) deltas self.csim.tedge - self.csim.pedge. The problem is then that the approximation errors are accumulated and over long simulations, the self.n within tqdm diverges from the actual simulation time. So then this happens:

[INFO][tb_cdrng.cdrng]: 2020-07-02 04:49:40: Simulating from 0.0 to 2.000m.

100%|███████████████████████████████▉| 1.999ms/2.000ms [01:36<00:00, 47.13s/ms]
/Users/colinww/anaconda3/lib/python3.7/site-packages/tqdm/std.py:484: TqdmWarning: clamping frac to range [0, 1]
  charset=Bar.ASCII if ascii is True else ascii or Bar.UTF)
100%|████████████████████████████████| 2.008ms/2.000ms [01:37<00:00, 48.31s/ms][INFO][tb_cdrng.cdrng]: 2020-07-02 04:51:17: Simulation paused at 2.000m.

So in this case, all the timestep approximation errors were biased slightly positive, so eventually tqdm thinks we have exceeded our total (by about 0.008 / 2.0). To work around this, I have changed my update to look like this:

while SimState.SIM_RUN == self.csim.cstate:
    bar.n = self.csim.tedge - sim_tstart
    bar.update(0)

This seems to work. But is this the "official" way to do this? Essentially I want an update method that takes in an absolute position, rather than a delta increment. Does this exist? Thank you!

[colinww]

I should add that I don't think this is a bug with tqdm, since it is more related to my specific needs. But I think in general, if a floating-point total is supported, it is dangerous to use the delta-update method, since a running sum of floating point numbers (and comparisons with floats, etc) can lead to all sorts of imprecisions.

[sigvaldm]

I second @colinww. In fact, I think there should be a method set_progress() or something, which sets the current progression instead of just updating by a number.