nb_available(open(fn)) != nb_available(Filesystem.open(fn))

[samoconnor]

Is this expected behaviour?

Version 0.5.0-dev+1968 (2016-01-02 23:28 UTC)

julia> f = open("/tmp/foo.csv")
IOStream(<file /tmp/foo.csv>)

julia> nb_available(f)
0

julia> f = Base.Filesystem.open("/tmp/foo.csv", Base.Filesystem.JL_O_RDONLY)
Base.Filesystem.File(true,RawFD(18))

julia> Base.Filesystem.nb_available(f)
6

[vtjnash]

nb_available is the number of bytes that can be read from the fd without a syscall (or other operation that might block). The correct answer is 0. It looks like the libuv File wrapper is not consistent here however.

[samoconnor]

The manual entry says:

nb_available(stream)
Returns the number of bytes available for reading before
a read from this stream or buffer will block.

Given that /tmp/foo.csz is a 6-byte local file, it seems fair to assume that if you do a read it will return the 6 bytes without having to block. The definition in Filesystem.jl is nb_available(f::File) = filesize(f) - position(f). This makes it pretty clear that @stevengj intended this behaviour.

[samoconnor]

@stevengj, do you have any comment on this? I'd like to add a test that checks consistency of nb_available across different stream types to this test.

[samoconnor]

@stevengj bump...

[samoconnor]

Note change to nb_available(File) : https://github.com/JuliaLang/julia/pull/15090

[stevengj]

In general, it doesn't seem possible for nb_available to return the number of bytes that can be read from an arbitrary stream (e.g. in the case of a pipe this is not known in advance). So, there are two options here:

• For the specific case of a file stream, return the same thing as nb_available(::File). This has the disadvantage of requiring us to work around libuv.
• Leave the behavior as-is.

In either case, better documentation is required.

[vtjnash]

Filesystem.File still appears to be wrong here:

julia> f = Base.Filesystem.open("VERSION", Base.Filesystem.JL_O_RDONLY)
Base.Filesystem.File(true, RawFD(19))

julia> bytesavailable(f)
11

julia> f = open("VERSION", "r")
IOStream(<file VERSION>)

julia> bytesavailable(f)
0