alexsilva / supervisor

Supervisor process control system for Windows
http://supervisord.org
Other
118 stars 26 forks source link

use _winapi.CreatePipe as stdout is really slow #39

Closed museless closed 3 years ago

museless commented 3 years ago

Python Version: 3.8 Environment: Win Server 2016 / Visual Studio 2019 Package Version: supervisor-win==4.6.0

#include <iostream>
#include <chrono>
#include <string>

using std::chrono::high_resolution_clock;
using std::chrono::microseconds;
using std::chrono::duration_cast;
using std::string;

void test_print(void)
{
    for (int idx = 0; idx < 100000; ++idx) {
        std::cout << "Hi!" << std::endl;
    }
}

int main()
{
    auto start = high_resolution_clock::now();
    test_print();
    auto span = duration_cast<microseconds>(high_resolution_clock::now() - start);

    std::cout << span.count() << std::endl;

    return  0;
}

When I click the exe to run, it run 10-15 seconds. But when I put it on supervisord, it run at lease several minutes.

I track down the code. (C1)

# process.py line 392
    @classmethod
    def execute(cls, filename, argv, **kwargs):
        """Runs a new process and returns its reference"""
        redirect_stderr = kwargs.pop('redirect_stderr', False)
        kwargs.update(dict(
            # Prevents the supervisor from inheriting the signal when expected by the process.
            creationflags=subprocess.CREATE_NEW_PROCESS_GROUP,
            stdout=subprocess.PIPE,
            stdin=subprocess.PIPE,
            bufsize=1024 * 2,  # 2Kb
        ))

stdout is equal to subprocess.PIPE, and it keep running to subprocess.py (C2)

            # line 1163
            if stdout is None:
                c2pwrite = _winapi.GetStdHandle(_winapi.STD_OUTPUT_HANDLE)
                if c2pwrite is None:
                    _, c2pwrite = _winapi.CreatePipe(None, 0)
                    c2pwrite = Handle(c2pwrite)
                    _winapi.CloseHandle(_)
            elif stdout == PIPE:
                c2pread, c2pwrite = _winapi.CreatePipe(None, 0)
                c2pread, c2pwrite = Handle(c2pread), Handle(c2pwrite)
            elif stdout == DEVNULL:
                c2pwrite = msvcrt.get_osfhandle(self._get_devnull())
            elif isinstance(stdout, int):
                c2pwrite = msvcrt.get_osfhandle(stdout)
            else:
                # Assuming file-like object
                c2pwrite = msvcrt.get_osfhandle(stdout.fileno())
            c2pwrite = self._make_inheritable(c2pwrite)

It seems like subprocess create a pipe to keep output from child process.

I not familiar with windows api.

And I try to change stdout from C1, DEVNULL will raise error, None can make its normal, but all output write to supervisord console.

Is there any solution to make it faster.

Thx.

museless commented 3 years ago

Test ini here.

[program:Test]
command=C:/Data/code/Test/x64/Release/Test.exe
directory=C:/Data/code/Test/x64/Release
user=root
autostart=false
autorestart=false
pidfile=C:/Data/var/run/test.pid
stderr_logfile=NONE
stdout_logfile=NONE
alexsilva commented 3 years ago

In the default configuration the reading of data takes place every 0.5 seconds. try changing this to a smaller number.

delaysecs=0.5

museless commented 3 years ago

Thx, it work.

Seems like a new victim about windows don't have epoll.