No newlines printed when output is not a terminal

[avnf]

SUMMARY

I want to run dool over ssh on remote nodes. But when output is not a terminal, no newline is printed

ISSUE TYPE

• Bug Report / - Feature Idea

DOOL VERSION

Dool 1.3.3

OS / ENVIRONMENT

Linux Ubuntu 22.04

STEPS TO REPRODUCE

./dool-orig/dool |cat

It should be like normal dool, only without color; with newlines. (some remote ssh-like tools only flush on newline)

EXPECTED RESULTS

$ ./dool-orig/dool
Using default plugins: cpu, disk, net, load
┄┄total┄cpu┄usage┄┄┬┄dsk/total┄┬┄net/total┄┬┄┄┄load┄avg┄┄┄┬┄┄┄┄┄system┄┄┄┄
usr sys idl wai stl│ read  writ│ recv  send│ 1m   5m  15m │      time
  0   0 100   0   0│ 199k  116k│   0     0 │0.21 0.87 0.99│Sep-11 11:56:51
  0   0 100   0   0│   0     0 │3264b   10k│0.21 0.87 0.99│Sep-11 11:56:52
  0   0 100   0   0│   0     0 │ 528b 3120b│0.21 0.87 0.99│Sep-11 11:56:53

ACTUAL RESULTS

./dool-orig/dool |cat
Using default plugins: cpu, disk, net, load
┄┄total┄cpu┄usage┄┄┬┄dsk/total┄┬┄net/total┄┬┄┄┄load┄avg┄┄┄┬┄┄┄┄┄system┄┄┄┄
usr sys idl wai stl│ read  writ│ recv  send│ 1m   5m  15m │      time
  0   0 100   0   0│ 199k  116k│   0     0 │1.56 1.33 1.14│Sep-11 11:53:55  0   2  98   0   0│   0     0 │ 528b 4976b│1.56 1.33 1.14│Sep-11 11:53:56  0   1  98   0   0│   0     0 │1896b 2680b│1.56 1.33 1.14│Sep-11 11:53:57^C

$ ssh localhost ./dool-orig/dool
Using default plugins: cpu, disk, net, load
┄┄total┄cpu┄usage┄┄┬┄dsk/total┄┬┄net/total┄┬┄┄┄load┄avg┄┄┄┬┄┄┄┄┄system┄┄┄┄
usr sys idl wai stl│ read  writ│ recv  send│ 1m   5m  15m │      time
  0   0 100   0   0│ 199k  116k│   0     0 │1.04 0.92 1.00│Sep-11 11:57:49  0   0  87  13   0│   0     0 │ 528b 4976b│3.60 1.45 1.17│Sep-11 11:57:50  0   0  85  15   0│   0   426k│ 528b 1840b│3.60 1.45 1.17│Sep-11 11:57:51^C

# nocolor don't help

$ ./dool-orig/dool --nocolor --noupdate|cat
Using default plugins: cpu, disk, net, load
┄┄total┄cpu┄usage┄┄┬┄dsk/total┄┬┄net/total┄┬┄┄┄load┄avg┄┄┄┬┄┄┄┄┄system┄┄┄┄
usr sys idl wai stl│ read  writ│ recv  send│ 1m   5m  15m │      time
  0   0 100   0   0│ 198k  116k│   0     0 │20.8 6.62 2.96│Sep-11 11:58:44  0   0  85  15   0│   0     0 │1344b 5504b│20.8 6.62 2.96│Sep-11 11:58:45  0   0  85  15   0│   0     0 │1896b 2680b│20.8 6.62 2.96│Sep-11 11:58:46^C

[scottchiefbaker]

Good catch, and good troubleshooting. You made this easy to recreate which was key to fixing the issue. I believe I landed the fix in ddf641f. Can you please test that and let me know if that solves your problem?

It's worth noting that ssh domain.com | dool --color should have always worked. We just had a weird logic bug that was not triggering \n when not attached to a TTY.

[avnf]

Thank you for quick response, this change from ddf641f fixed output when output is not a terminal but a "cat":

$ ./dool |cat
Using default plugins: cpu, disk, net, load
┄┄total┄cpu┄usage┄┄┬┄dsk/total┄┬┄net/total┄┬┄┄┄load┄avg┄┄┄┬┄┄┄┄┄system┄┄┄┄
usr sys idl wai stl│ read  writ│ recv  send│ 1m   5m  15m │      time
  0   0  99   0   0│   0     0 │3168b   16k│4.50 21.0 26.8│Sep-13 02:49:06
  0   0  99   0   0│   0     0 │7168b   14k│4.50 21.0 26.8│Sep-13 02:49:07

And yes, ssh hostname dool --color also worked

With ssh it also works fine, 1 line per second for default mode. But when interval is greater than one, there are updates to the line (probably with backspace). Is it tunable, for the case when somebody wants just 1 line per interval and nothing else (for example when "script" logging of console is active")? Just like old classic linux vmstat, iostat, sar -n DEV.

I also tested this is clustershell "clush": clush -w nodename dool/dool 1 and it don't print 1 line per second. It buffers 3 lines and output them once. Same with "python -u dool/dool 1" Clustershell has some output processing, it puts node name before every flushed group of strings probably

PS One more minor difference from vmstat. Vmstat outputs one line with end of line \n, and then waits for interval. When I press Ctrl-C using ssh for remote run, cursor position is column 0. With dool, cursor is positioned at the end of line, and no \n printed immediately. When I press Ctrl-C, my original shell prints its prompt to the right of last line, not in newline.

user@hostname:~$ ssh hostname dool 
...
  1   1  99   0   0│   0     0 │1584b   16k│1.93 3.57 13.6│Sep-13 03:00:30
  1   1  99   0   0│   0  2687k│  10k   26k│2.26 3.61 13.6│Sep-13 03:00:31^Cuser@hostname:~$

[scottchiefbaker]

Interesting... I learned something about dool I didn't know. We have the --noupdate option already which does exactly what you want. That causes dool to only output on the interval, and not on the intermediate lines. It wasn't working though, so I just fixed it in ae37f2b. Can you test that and let me know?

Also worth noting we have --output foo.csv to output raw numbers to a CSV if you want to run more detailed reports yourself.

[scottchiefbaker]

Did this last commit fix the issue for you?

[avnf]

Yes, with ae37f2b commit ssh dool 3 works like vmstat 3; now no extra "update" lines are generated or printed. clush has same extra buffering for vmstat too.

[scottchiefbaker]

Excellent