schveiguy / io

Core I/O functionality
Boost Software License 1.0
25 stars 9 forks source link

Provide a way to open standard handles #14

Closed schveiguy closed 4 years ago

schveiguy commented 6 years ago

Opening the standard handles on Windows and Posix is drastically different with low-level i/o. For D and C, you just use stdout/stdin/stderr, which are FILE * (or phobos File objects).

In io, you need to wrap the low-level handles/descriptors as File objects. We should provide a way to open a standard handle (maybe a D getStdHandle that forwards to the windows version, or returns 0, 1, 2 on Posix) to make cross-platform usage of the standard handles convenient.

We also may want to provide a way to use standard handles with a global, to prevent accidental closure of the handles, since IOs close themselves compulsively.

MartinNowak commented 6 years ago

This seem similar to #7. Will have to think about stdout/stdin/stderr a bit more.

schveiguy commented 6 years ago

This report comes out of migrating my iopipe examples to work on both Windows and Posix. Currently, code looks like this:

openDev(0).bufd....

Where openDev (was posix only) is now a wrapper to ioObject(File(fd))

But now it must work on Windows too, and it's not nearly as straightforward.

Indeed, I've added a deprecation message saying to use std.io, but I don't have a good replacement for this.

So this issue really is just "what's the best way to portably do this?" more than #7 which is focused on how to wrap e.g. stdin without closing it for other cases.

schveiguy commented 5 years ago

I think a "initialized on first use" set of handles would be useful to have in the driver. Just abstract the mechanism of getting the default io objects. Not sure, you might need to get only the handles themselves, and let the user wrap with the appropriate type (e.g. stdin may be a socket, and you may want to open it that way).

jondegenhardt commented 5 years ago

I put together a simple app to do performance comparison between I/O mechanisms in Phobos, iopipe/std.io, and a couple things in tsv-utils Mostly testing byLine style I/O so far. The iopipe/std.io pair is the fastest, especially on skinny files. Very promising. It'd be great to pick up this pair and use them at some point. I also want to experiment more with the output side, where I believe there are performance gains to be had. (Test app: dcat-perf, Benchmarks report: Issue #1)

This very simple exercise raised a few questions, the first one on the topic of this issue: standardized access to stdin, stderr, and stdout. It is clunky to be accessing them via the raw integer handles. This is important to address and no doubt will be.

Bigger picture - At this point I don't understand how std.io facilities are intended to fit into the overall library picture. In particular:

If you can help me understand the larger picture I can provide some better targeted questions and feedback. e.g. "What happens to basic facilities like writeln?". Better context will help formulate more worthwhile questions.

jondegenhardt commented 5 years ago

I ran a benchmark using a simple buffering scheme on top of io output to standard output. Quite an improvement over the unbuffered version. I only tried standard output, don't know if its more general and applies to output to files, etc.

Benchmark here: dcat-perf Issue 4

WebFreak001 commented 4 years ago

can be closed because of v0.3.0?

schveiguy commented 4 years ago

Yep, thanks for reminding me.