Add a nice spinning bar

[LemonBoy]

44

Less is more (?)

[kenorb]

It's possible to make it compatible with CI output (doesn't need to show progress there, but at least it won't generate thousands of one-char lines)? Using pv works fine for CI.

It also fails with:

The log length has exceeded the limit of 4 MB (this usually means that the test suite is raising the same exception over and over).

[LemonBoy]

Travis's terminal doesn't interpret any escape sequence nor a simple \b so I guess the only solution is to make the bar spin at a lower rate, maybe after 10000 ticks or so

[kenorb]

Please check test branch, using pv seems to work.

Tested by: cat /dev/urandom | pv -N 'Progress' > /dev/null.

Travis CI:

Local (same branch test, just run make):

We don't need progress in CI, just on local. It seems CI somehow detects the progress bar and it doesn't printing it in each line.

[LemonBoy]

Better ?

[kenorb]

I think the point is to print anything, so Travis won't time out because lack of any output for longer than 10 minutes.

In branch test I've run:

cat /dev/urandom | pv -N 'Progress' > /dev/null

which shows the progress locally and it seems it was showing on Travis as well, but not in real time, just after finish:

cat /dev/urandom | pv -N 'Progress' > /dev/null
 Progress: 40.1GB 0:48:17 [14.6MB/s] [  <=>  

But at least it didn't stopped, because of no output was received for longer than 10 minutes.

See: #119190362

[LemonBoy]

Some text is already printed in between timeframes conversion so the problem here is to let travis know the script isn't dead during the conversion phase, the performance fixes might help here since it takes less than 10m to convert a single set, otherwise the only other option I see is to rate-limit the progressbar

[LemonBoy]

Also, see https://docs.travis-ci.com/user/common-build-problems/ about travis_wait

[kenorb]

Text between timeframes is not enough, since M1 is taking the longest time, probably more than 30 minutes.

[LemonBoy]

Ayy, is this ok ?

[LemonBoy]

Green :green_apple:

[kenorb]

The previous version was better, with the progress bar. I'm investigating currently if the progress bar is possible with CI.

[LemonBoy]

Nope, most projects just disable the bars alltogether when the 'CI' env variable is detected since Travis doesn't support any kind of escape sequence

[kenorb]

I did simple test in Python and it seems it works fine.

Build: 119308690 File: progress-bar.py Code: http://stackoverflow.com/a/3160819/55075 Repo: travis-test (you can fork it and test CI on your side)

I think you just need to send return of the line (\b) and flush the stdout.

[LemonBoy]

ehh, the original version had no such luck, is a spinning bar ok ?

[kenorb]

The spinning is fine (e.g. every 1% as per issue description), but it shouldn't generate thousands of lines (just one line with any progress bar). If we know the total of ticks, we can print how many ticks are of total.

[LemonBoy]

You can't know the number of ticks since the Input iterator is lazy and you'd end up reading the whole dataset twice (once for len() and once for the actual elaboration), so the best you could have is the crazy spinning bar :)

[kenorb]

Ok, spinning bar is fine or e.g. elapsed time, or just how many ticks were already proceed.

[LemonBoy]

It looks like travis doesn't like multiple \b

[kenorb]

Probably because there are too many characters printed (>4MB). Can you do it each 2 seconds or every 100k or 1mln ticks? If you're printing it on each tick, the whole year could have 17mln of ticks.

[kenorb]

Almost.

Sometimes it's replacing some previous characters, sometimes it's not.

But it's saying:

The log length has exceeded the limit of 4 MB (this usually means that the test suite is raising the same exception over and over).

Probably too many bars in short amount of time.

[LemonBoy]

Slightly better

[kenorb]

If this can't be done in a better way, pity. At least it works in terms of outputting something.