Wishlist: unbuffered I/O

[GoogleCodeExporter]

When using additional tools which parse geotoads' output in real-time, it is 
nice to have unbuffered I/O. Otherwise the operating system / shell might 
choose to buffer output until a certain number of bytes is reached before it is 
handed to the next process in the pipe. Adding $stdout.sync = true somewhere at 
the beginning fixes this issue.

Index: geotoad.rb
===================================================================
--- geotoad.rb  (Revision 1208)
+++ geotoad.rb  (Arbeitskopie)
@@ -7,6 +7,7 @@
 $LOAD_PATH << (File.dirname(__FILE__.gsub(/\\/, '/')) + '/' + '..')

 $isRuby19 = false
+$stdout.sync = true

 if RUBY_VERSION.gsub('.', '').to_i < 180
   puts "ERROR: The version of Ruby your system has installed is #{RUBY_VERSION}, but we now require 1.8.0 or higher"

I'm using a Python script which parses the output to display a GUI dialog with 
a progress bar. If output is buffered, the progress bar is very jumpy because 
the GUI only receives updates when the I/O buffer has been filled completely.

Original issue reported on code.google.com by jan.reuc...@gmail.com on 18 Apr 2013 at 7:09

[GoogleCodeExporter]

Nice idea... as far as I know you're the only one piping GeoToad's output 
somewhere else. What about the performance penalty? (In particular, what would 
happen in the standard setting with --verbose set?)
Would $stdout.flush() after writing the progress line (last puts in 
interface/progressbar.rb) do the trick, too?

Changing from "Defect" to "Enhancement"...

Original comment by Steve8x8 on 19 Apr 2013 at 8:51

• Added labels: Type-Enhancement
• Removed labels: Type-Defect

[GoogleCodeExporter]

What about running
   ruby -e '$stdout.sync=true' geotoad.rb
in your python wrapper as a workaround?

Original comment by Steve8x8 on 19 Apr 2013 at 9:40

[GoogleCodeExporter]

Regarding performance: you'll only notice the difference if you pipe the 
verbose output to a file, because direct console I/O is usually unbuffered. 
Flushing stdout after writing the progress line would be sufficient. But if 
your workaround also does the trick, this would be sufficient for me. I'll test 
it.

Original comment by jan.reuc...@gmail.com on 20 Apr 2013 at 10:22

[GoogleCodeExporter]

Hmm, that doesn't work:

  -e command     Specifies script from command-line while telling Ruby not
                    to search the rest of the arguments for a script file
                    name.

So geotoad.rb is not executed.

Original comment by jan.reuc...@gmail.com on 20 Apr 2013 at 10:31

[GoogleCodeExporter]

Digging through the command line options, I found a similar solution. I wrote a 
short ruby script which I now load using ruby -r unbuffered.rb ...

unbuffered.rb:

$stdout.sync=true
puts "Enabled unbuffered I/O"

So there's no need for a modification in geotoad. You can close this issue. 
Thanks for the tip!

Original comment by jan.reuc...@gmail.com on 20 Apr 2013 at 10:44

[GoogleCodeExporter]

Thank you - for pointing me to the feature I had known for Perl but never found 
for Ruby... and unfortunately mixing -e "..." and a script name wouldn't work, 
of course.
It's a pity that the request for a -u option to the ruby invocation (with 
similar effect as it has in Python) has never been considered.

Closing...

Original comment by Steve8x8 on 20 Apr 2013 at 11:22

• Changed state: WontFix

[GoogleCodeExporter]

There has been a second request, this time from a Windows user. (As there's no 
accessible ruby invocation, the solution presented above won't work.)
Thinking about another option (-U = --unbufferedOutput), and enabling that by 
default if running from within a Windows exe ... any thoughts?

Original comment by Steve8x8 on 20 Jan 2014 at 12:30

• Changed state: Accepted

[GoogleCodeExporter]

Absolutely ok for me.

Original comment by jan.reuc...@gmail.com on 20 Jan 2014 at 7:18

[GoogleCodeExporter]

So at the moment, this would be a two-stage process:

(a) Add another option (-U, to my surprise, is still unused) to set output to 
unbuffered (actually, there are a few "print" statements in the code, so it'll 
not be line-buffered - if that makes a difference). That's simple.

(b) Force output to unbuffered _even if the user hasn't explicitly chosen this_.
This one is causing me headaches: 
Would it be OK to default to unbuffered *if* $stdout.tty? returns false, always?
(What about the performance penalty caused by frequent flush operations? What 
about write wear of SSDs?)
Is automatic detection (of whatever) ever required? For which use cases? I 
cannot imagine a use case where GeoToad output gets passed to another process, 
without the chance of adding a runtime flag, not even in a Windows environment 
- as I'm not a Windows user. -> Tell me more about your use case! (Martin, have 
you made it here?)

Attached are my quick-and-dirty shots at the 2 stages outlined above...

Original comment by Steve8x8 on 21 Jan 2014 at 9:13

Attachments:

• 267a-unbuffered_io.diff
• 267b-sync_windows.diff

[GoogleCodeExporter]

Patch (a) has been submitted. 
(b) will be postponed, and may be dropped completely - unless someone shows me 
there's a use case for TUI (= option-less invocation) in a wrapper context.

For the time being, a -U option added to the command-line will switch to 
unbuffered output after a few lines (immediately after command-line option 
parsing). This should be enough (in my limited imagination)

Original comment by Steve8x8 on 24 Jan 2014 at 1:01

• Changed state: FixAvail

[GoogleCodeExporter]

Included in 3.20.1, released a few minutes ago.

Original comment by Steve8x8 on 25 Feb 2014 at 11:26

• Changed state: Fixed