Zombie processes with loop.subprocess_exec

[blueyed]

I've noticed that if the deoplete main process exits with an error it will stay around as a Zombie process.

I couldn't reproduce it with a standalone script outside of Neovim, so I assume there might be something in Neovim's python-client that might cause it.

Deoplete's source: https://github.com/Shougo/deoplete.nvim/blob/a80fd5267e978ab86a1e30975be9457df3394646/rplugin/python3/deoplete/parent.py#L92-L98

Test script:

import asyncio
from functools import partial

class Process(asyncio.SubprocessProtocol):

    def __init__(self, plugin):
        with open('/tmp/deoplete-2.log', 'a') as f:
            f.write('init\n')
        self._plugin = plugin
        # self._vim = plugin._vim

    def connection_made(self, transport):
        self._plugin._stdin = transport.get_pipe_transport(0)
        with open('/tmp/deoplete-2.log', 'a') as f:
            f.write('connection made: %r\n' % transport)
        print('connection made: %r\n' % transport)

    def connection_lost(self, exc):
        with open('/tmp/deoplete-2.log', 'a') as f:
            f.write('connection lost: %r' % exc)

    def pipe_data_received(self, fd, data):
        print('pipe_data_received: %d, %r' % (fd, data))
        with open('/tmp/deoplete-2.log', 'a') as f:
            f.write('pipe_data_received: %d, %r\n' % (fd, data))
        if fd == 1:
            unpacker = self._plugin._unpacker
            unpacker.feed(data)
            for child_out in unpacker:
                self._plugin._queue_out.put(child_out)
        else:
            # self._vim.error
            raise Exception('pipe_data_received: unexpected output on %d: %s' % (fd, data))

    def process_exited(self):
        print('process_exited')
        with open('/tmp/deoplete-2.log', 'a') as f:
            f.write('process exited: %r' % self)

class Plugin:
    pass

plugin = Plugin()

loop = asyncio.get_event_loop()
task = loop.create_task(
    loop.subprocess_exec(
        partial(Process, plugin),
        'python3', 'meh',
        # stdout=None,
        # stderr=None,
        cwd='/tmp'))
loop.run_forever()
# loop.run_until_complete(task)
loop.close()

Needs further investigation (also from my side), but maybe you have some pointer(s) already?

/cc @shougo

[justinmk]

If the deoplete main process exits with an error it will stay around as a Zombie process.

How can it exit yet "stay around"? Which process exactly?

[blueyed]

@justinmk AFAIK a process becomes a zombie process for a parent's wait. So it looked to me like Neovim's loop handling might never trigger that wait for some reason, but I have not investigated further, and unfortunately could not reproduce it using a simpler script back then.