search

golang / go

The Go programming language
https://go.dev
BSD 3-Clause "New" or "Revised" License
121.07k stars 17.36k forks source link

os: support runtime poller with os.File on Windows #19098

Open ianlancetaylor opened 7 years ago

ianlancetaylor commented 7 years ago

In order to use the runtime poller with os.File on Windows, the file or pipe will have to be opened with FILE_FLAG_OVERLAPPED. We shouldn't do that unconditionally unless we can remove it if the program calls the Fd method. Programs currently expect the Fd method to return a handle that uses ordinary synchronous I/O.

gopherbot commented 7 years ago

CL https://golang.org/cl/36800 mentions this issue.

rasky commented 7 years ago

ReOpenFile() can do just that, adding/removing FILE_FLAG_OVERLAPPED (https://msdn.microsoft.com/en-us/library/aa365497(VS.85).aspx)

polarina commented 7 years ago

Overlapped reads on Windows may run synchronously under some conditions. Files using NTFS compression can not be read asynchronously, for example.

See https://support.microsoft.com/en-us/help/156932/asynchronous-disk-i-o-appears-as-synchronous-on-windows

ianlancetaylor commented 7 years ago

Thanks. For our purposes it doesn't matter if I/O is sometimes synchronous. The effect will be that the I/O operation will tie up a thread. That is not ideal, but it is no different than any other blocking syscall.

jstarks commented 7 years ago

There's no way to flip the bit on an existing file handle, and ReOpenFile will only work properly for real local files, only if the file does not have the delete on close bit set (I think), and only if they have been opened with compatible sharing flags. And of course Fd() uintptr does not provide a mechanism to return an error if something goes wrong.

I'm afraid this may be impossible to achieve with the current Windows API.

This is probably not such a shame for normal files opened for cached IO, although recent versions of Windows do perform cached reads asynchronously in some cases. But it certainly is limiting for named pipes, files opened with FILE_FLAG_NO_BUFFERING, and other Windows file handle types that reliably perform IO asynchronously.

Maybe this is something that we can improve in Windows. But even if we do, I think for Windows there would still be a lot of value in providing direct access to the runtime poller. There are several APIs that take OVERLAPPED structures that don't fit into os.File (ConnectNamedPipe, MergeVirtualDisk, DeviceIoControl, etc.), and it would be ideal to be able to use the existing poller to avoid blocking threads on these APIs.

That seems at odds with some of the discussion in #18507, unfortunately.

qiulaidongfeng commented 5 months ago

CL has been merged, is it still necessary to open this issue?

qiulaidongfeng commented 5 months ago

cc @golang/windows This have OS-Windows label , CL has been merged. Is there anything else to be done here?

crvv commented 5 months ago

The merged CL(36800) updated this issue. It didn't fix this issue. According to the discussion above. This issue can't be fixed because of the Fd method. My personal opinion is that Fd usages should be replaced by SyscallConn. And Fd can be deprecated and removed.