Open fzhinkin opened 2 months ago
Very welcomed enhancement. I was considering implementing this kind of project, and having stdin and stdout for Native implemented here would definitely make my life easier.
stderr would also be useful.
On ~native~ POSIX, being able to wrap a raw sink / raw source around any file descriptor would be convenient. The biggest downside is exposing the type-unsafe FD parameter as just an Int
or whatever. But presumably it would be the implementation you'd use for stdin and stdout on native anyway, so having it public saves people from getting it wrong when they want to read/write some other FD.
we can return an always-exhausted
Source
and aSink
writing toconsole.log
.
console.log/error
won't work here as is: RawSink
is dealing with raw bytes, not a strings. We can, probably, try to reinterpret these bytes as string ourselves, and then emit the result into console.log
, but I'm not sure if it's worth the effort.
Another option to consider: if these stdin/out/err Source
/Sinks
should be buffered or Raw
.
Buffered sinks and sources could reduce chances of closing application-wide streams/files accidentally by writing:
stdoutSink().buffered().use {
it.writeString("Goodbye, /proc/self/fd/1")
}
Instead, it would be:
val out: Sink = stdoutSink()
out.writeString("Hey!")
out.flush()
console.log/error won't work
For the browser I'd just avoid implementing it for the first iteration, let's see how usable the outcome is first.
For Node.js you can use process.stdin/stdout/stderr
with fs.readSync
I suppose, although for this kind of streams it would cool to have a suspending variant at some point.
Yeap, for NodeJs everything works with existing FileSink
and FileSource
implementations (assuming their constructors will accept FD
instead of a file path).
It looks like you'll just have to create secondary constructors.
I see that the Node.js version uses readFileSync
. You could switch to readSync
and keep your own Buffer
instance around, which would also avoid having to assert (!!
) it.
Another option to consider: if these stdin/out/err Source/Sinks should be buffered or Raw.
When re-doing readln
, we explicitly ruled out buffering to avoid interference with other platform sources (e.g. System.in
or System.console()
) -- the idea was to be able to have independent readLine
calls between various sources.
The argument mostly holds if we are going to provide top-level access like Source.stdin
On a side note, we might be able to finally address the annoying windows console encoding issue
The argument mostly holds if we are going to provide top-level access like
Source.stdin
Then we have to use some alternative buffering implementation as the current one prefetch data eagerly.
Source.stdin
could be Source
, not a RawSource
. And we can use either new implementation, or parameterize RealSource
to buffer only requested number of bytes from the underlying input stream / fd. That'll prevent unwanted interference with other platform sources.
However, that'll work until the first .buffered()
call on Source.stdin
as a newly created buffered source will buffer eagerly. And since we encourage users to use RawSource
where possible, Source.stdin
will likely be wrapped into yet another buffered source somewhere, where its origin will be unknown:
fun parseJson(source: RawSource) {
val buffered = source.buffered()
...
}
// jq.kt
fun main(args: Array<String>) {
val inputFileName = getFileName(args)
if (inputFileName == "-") {
parseJson(Source.stdin)
} else {
SystemFileSystem.source(Path(inputFileName)).use {
parseJson(it)
}
}
}
Yet another issue is that Source.exhausted
needs to read at least a single byte from an underlying source, so it will always cause interference.
After some discussions, it seems like the fact that stdin-backed Source
will mess up with readln
is not a problem (or, to be more precise, it's not a problem we can solve for Source
):
RawSource
won't have this problem, it'll read requested number of bytes only;StandardInputSource.buffered()
;It's worth mentioning that such a source won't replace something similar to Java's Console
class and if interactive I/O through a virtual terminal is required, some other interface should be introduced.
The same considerations are also applicable to stdout/err-backed Sinks.
Other questions, however, remain open:
System.in
stream, should previously created source continue reading from the old stream, or should it switch to a newly set stream?On Java, should these sinks/sources follow changes of corresponding System.in/out/err streams?
I'm a pretty strong "no" on this.
The analogous behavior on the JVM would be calling System.out
to obtain an OutputStream
, and wanting that OutputStream
instance to change where it writes to if someone calls System.setOut
.
This problem isn't limited to the JVM. If you close the stdin FD on native, the next opened file will get the "standard" STDIN FD number.
This problem isn't limited to the JVM. If you close the stdin FD on native, the next opened file will get the "standard" STDIN FD number.
It seems to be the same as
follow changes of corresponding System.in/out/err streams
So if we want not to follow, on native a duplicate FD is needed. As a bonus, duplicating an FD will solve an issue with accidentally closing a stdin/out/err stream.
Some applications may require reading from stdin or writing to stdout, but there's no built-in factories/functions returning corresponding
Sink
/Source
implementations.It should be trivial to provide them for all targets except JS running in the browser. For the latter, we can return an always-exhausted
Source
and aSink
writing toconsole.log
.