Optionally write directly to pipe on Windows

[qwerty12]

On Windows, using io.open on the named pipe works and Lua can write to it directly without needing to start an external program. This isn't asynchronous unlike subprocess, however, and the write might block the script for too long if the thumbnail process doesn't acknolwedge the request quickly, leading mpv to terminate the script, so this is under a disabled-by-default option. I haven't noticed any issues personally, but if there were, I guess it would be more noticable if trying to thumbnail a non-local video. In my case, I'll take the risk, as starting many cmds on my system is a little slow.

If a callback is given to run, then the pipe is opened for reading too and mpv's reply is stored in the stdout member of a table similar to one subprocess would return to the callback.

I couldn't work out how to reuse the handle (assuming it's possible) when I was experimenting with using LuaJIT FFI to call WriteFile directly so here it's just closed and opened as necessary. This also unfortunately doesn't work on Linux - the only ways I know to do the same thing there would be to use the LuaSocket or luaposix modules, or if built against it (it is on Arch), LuaJIT's built-in ffi module.

[hooke007]

The more severe lag issue here.

ref https://github.com/tomasklaen/uosc/issues/258

[qwerty12]

Ah, reading the reply is slow. I apparently didn't notice before opening this, my apologies.

Since thumbfast currently doesn't actually care about mpv's reply, try this: replace the open_for_reading line with local open_for_reading = false. It (hopefully) shouldn't be as bad then.

[hooke007]

Ah, it worked but would be more easily freezed than master branch... https://github.com/po5/thumbfast/issues/11

[po5]

Wow, I never thought of that. Would love to make this the default if we can figure out a way to do it async.

Maybe add_timeout(0) + mp.utils.write_file or mp.utils.append_file, if that gets us real async within lua.

[qwerty12]

Maybe add_timeout(0) + mp.utils.write_file or mp.utils.append_file, if that gets us real async within lua.

I'm not sure but I think the write would still occur on the same thread the Lua script is being run on, so if that hangs, so does the Lua script.

I've pushed an update that eschews the Lua io functions in favour of the WinAPI functions through LuaJIT's ffi module. This allows the pipe handle to be put into non-blocking mode, which is the closest I can get. I've not measured anything, but run feels pretty quick to return. If use_lua_io (should be renamed, I guess) is enabled, this does require that mpv to be built against LuaJIT, but given that shinchiro's Windows builds do so, it's very unlikely a user has a Windows mpv build without LuaJIT, unless it's old or self-compiled. And in that case, the script will silently fallback to the subprocess method.

---

Another, separate approach: I did have a go at trying my hand at using the WriteFile in Overlapped mode, which is Windows' way of doing async I/O, but I don't like the result:

• there's a lot of code added
• I couldn't actually get Windows to trigger an async write once so the actual async code remains untested
• as far as I know, Windows' completion callbacks run on another thread, so the best workaround I could come up with was an mpv timer that checks the result of the write ASAP...
• I'm not entirely sure Lua is not garbage collecting by the time the timer is run? A reference to buf should probably be kept in the timer callback anyway

That diff for the sake of curiosity

```diff diff --git a/thumbfast.lua b/thumbfast.lua index b43d010..0d8d3cb 100644 --- a/thumbfast.lua +++ b/thumbfast.lua @@ -34,7 +34,10 @@ local options = { network = false, -- Enable on audio playback - audio = false + audio = false, + + -- Windows only: don't use subprocess to communicate with socket + use_lua_io = false } mp.utils = require "mp.utils" @@ -45,6 +48,68 @@ if options.min_thumbnails < 1 then options.min_thumbnails = 1 end +local winapi = {} +if options.use_lua_io then + local ffi_loaded, ffi = pcall(require, "ffi") + if ffi_loaded then + winapi = { + ffi = ffi, + C = ffi.C, + bit = require("bit"), + + -- WinAPI constants + FILE_SHARE_READ = 0x00000001, + FILE_SHARE_WRITE = 0x00000002, + FILE_SHARE_DELETE = 0x00000004, + GENERIC_WRITE = 0x40000000, + OPEN_EXISTING = 3, + FILE_FLAG_WRITE_THROUGH = 0x80000000, + FILE_FLAG_OVERLAPPED = 0x40000000, + FILE_FLAG_NO_BUFFERING = 0x20000000, + ERROR_IO_INCOMPLETE = 996, + ERROR_IO_PENDING = 997, + INVALID_HANDLE_VALUE = ffi.cast("void*", -1), + + -- don't care about how many bytes WriteFile wrote, so allocate something to store the result once + _lpNumberOfBytesWritten = ffi.new("unsigned long[1]"), + } + + if ffi.abi("64bit") then + ffi.cdef[[ + typedef unsigned __int64 ULONG_PTR; + ]] + else + ffi.cdef[[ + typedef unsigned long ULONG_PTR; + ]] + end + ffi.cdef[[ + typedef struct _OVERLAPPED { + ULONG_PTR Internal; + ULONG_PTR InternalHigh; + union { + struct { + unsigned long Offset; + unsigned long OffsetHigh; + } DUMMYSTRUCTNAME; + void *Pointer; + } DUMMYUNIONNAME; + + void *hEvent; + } OVERLAPPED, *LPOVERLAPPED; + + void* __stdcall CreateFileA(const char *lpFileName, unsigned long dwDesiredAccess, unsigned long dwShareMode, void *lpSecurityAttributes, unsigned long dwCreationDisposition, unsigned long dwFlagsAndAttributes, void *hTemplateFile); + bool __stdcall WriteFile(void *hFile, const void *lpBuffer, unsigned long nNumberOfBytesToWrite, unsigned long *lpNumberOfBytesWritten, LPOVERLAPPED lpOverlapped); + bool __stdcall CloseHandle(void *hObject); + unsigned long __stdcall GetLastError(); + bool __stdcall GetOverlappedResult(void *hFile, LPOVERLAPPED lpOverlapped, unsigned long *lpNumberOfBytesTransferred, bool bWait); + bool __stdcall CancelIoEx(void *hFile, LPOVERLAPPED lpOverlapped); + ]] + else + options.use_lua_io = false + end +end + local os_name = "" math.randomseed(os.time()) @@ -268,9 +333,56 @@ local function spawn(time) ) end +local function write_overlapped(command, callback) + local overlapped = winapi.ffi.new("OVERLAPPED") + local buf = command .. "\n" + -- open pipe with sharing enabled, so calls to this function while waiting for the result from a previous call don't fail + local hPipe = winapi.C.CreateFileA("\\\\.\\pipe\\" .. options.socket, winapi.GENERIC_WRITE, winapi.bit.bor(winapi.FILE_SHARE_READ, winapi.FILE_SHARE_WRITE, winapi.FILE_SHARE_DELETE), nil, winapi.OPEN_EXISTING, winapi.bit.bor(winapi.FILE_FLAG_WRITE_THROUGH, winapi.FILE_FLAG_NO_BUFFERING, winapi.FILE_FLAG_OVERLAPPED), overlapped) + if hPipe ~= winapi.INVALID_HANDLE_VALUE then + if not winapi.C.WriteFile(hPipe, buf, #buf + 1, winapi._lpNumberOfBytesWritten, nil) then + if callback and winapi.C.GetLastError() == winapi.ERROR_IO_PENDING then + local timer = nil + local start = mp.get_time() + timer = mp.add_periodic_timer(0, function() + local now = mp.get_time() + + if not winapi.C.GetOverlappedResult(hPipe, overlapped, winapi._lpNumberOfBytesWritten, false) then + if winapi.C.GetLastError() == winapi.ERROR_IO_INCOMPLETE then -- asynchronous write still in progress + if now - start < 2 then -- if it's been less than two seconds then check again when the timer is next hit + return + else -- if it's taking more than two seconds to write, cancel the write + winapi.C.CancelIoEx(hPipe, overlapped) + end + end + end + + -- if we're here, it's because the write succeeded, failed, or took more than two seconds + -- since run() runs any given callback unconditionally, do the same here + timer:kill() + winapi.C.CloseHandle(hPipe) + if callback then + callback() + end + end) + return -- running asynchronously, so run the callback and cleanup in the timer + end + end -- write completed synchronously or failed + winapi.C.CloseHandle(hPipe) + end + + if callback then + mp.add_timeout(0, callback) + end +end + local function run(command, callback) if not spawned then return end + if options.use_lua_io and os_name == "Windows" then + write_overlapped(command, callback) + return + end + callback = callback or function() end local seek_command ```

[hooke007]

LGTM. Great performance improvement I got.