jgehrcke / gipc

gevent-cooperative child processes and inter-process communication
https://gehrcke.de/gipc
MIT License
83 stars 13 forks source link

[Windows] Simple example breaking #19

Closed jgehrcke closed 6 years ago

jgehrcke commented 9 years ago

Originally reported by: phxrazdan (Bitbucket: phxrazdan, GitHub: Unknown)


Hi - I am trying to gipc work ... running python 2.7.9 and i can't get even the basic example to work without it throwing up (running in pycharm ed version)

I am trying to run exam # 3 https://gehrcke.de/gipc/ where it creates a simple duplex pipe. If I comment out sending any args ... the system just runs and of course does nothing. I must be breaking something very elementary for it not work but i am at a loss. I installed it with -U on pip so everything should be upgraded to the latest including greenlet and gevent.

Update: I downloaded the unit tests and 14 of the 57 failed. ...

And I get - the following ....

D:....\anaconda\python.exe C:/Users/../PycharmProjects/gipcexample/gipcexample.py Process _GProcess-1: Traceback (most recent call last): File "D:..\anaconda\lib\multiprocessing\process.py", line 258, in _bootstrap self.run() File "D:....\anaconda\lib\multiprocessing\process.py", line 114, in run self._target(*self._args, *self._kwargs) File "D:....\anaconda\lib\site-packages\gipc\gipc.py", line 361, in _child target(args, **kwargs) File "C:\Users....\PycharmProjects\gipcexample\gipcexample.py", line 27, in writer_process cend.put(time.time()) File "D:....\anaconda\lib\site-packages\gipc\gipc.py", line 953, in exit self[0].exit(exc_type, exc_value, traceback) File "D:...\anaconda\lib\site-packages\gipc\gipc.py", line 765, in exit "operation out of context." % self)) GIPCLocked: Context manager can't close handle <_GIPCReader_5564e3 fd: 5>. It's locked for I/O operation out of context. Traceback (most recent call last): File "C:/Users/arazdan/PycharmProjects/gipcexample/gipcexample.py", line 31, in main() File "C:/Users/..../PycharmProjects/gipcexample/gipcexample.py", line 14, in main assert pend.get() == "ACK" File "D:...\anaconda\lib\site-packages\gipc\gipc.py", line 855, in get msize, = struct.unpack("!i", self._recv_in_buffer(4).getvalue()) File "D:..\anaconda\lib\site-packages\gipc\gipc.py", line 810, in _recv_in_buffer "Most likely, the other pipe end is closed.") EOFError: Most likely, the other pipe end is closed.

Process finished with exit code 1


jgehrcke commented 8 years ago

Original comment by Arnon Yaari (Bitbucket: wiggin15, GitHub: wiggin15):


This reproduces for me with gevent 1.1.1 and gipc 0.6, on Windows only. gevent 1.0.2 works.

jgehrcke commented 8 years ago

Original comment by Jason Giedymin (Bitbucket: JasonG, GitHub: JasonG):


It works for me. I would recommend that you deploy a virtual environment which is clean from your system.

jgehrcke commented 6 years ago

@wiggin15 can you retry, can you maybe try current master?

wiggin15 commented 6 years ago

Still happens with master (0.7.0.dev0), gevent 1.2.1 or 1.3.6, Python 2.7 on Windows (tried example 3 on Windows Server 2008 and Windows 7):

Z:\Downloads>python ex3.py
Process _GProcess-1:
Traceback (most recent call last):
  File "C:\Python27\lib\multiprocessing\process.py", line 258, in _bootstrap
    self.run()
  File "C:\Python27\lib\multiprocessing\process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Python27\lib\site-packages\gipc-0.7.0.dev0-py2.7.egg\gipc\gipc.py", line 365, in _child
    target(*args, **kwargs)
  File "Z:\Downloads\ex3.py", line 25, in writer_process
    cend.put(time.time())
  File "C:\Python27\lib\site-packages\gipc-0.7.0.dev0-py2.7.egg\gipc\gipc.py", line 962, in __exit__
    self[0].__exit__(exc_type, exc_value, traceback)
  File "C:\Python27\lib\site-packages\gipc-0.7.0.dev0-py2.7.egg\gipc\gipc.py", line 772, in __exit__
    "operation out of context." % self))
GIPCLocked: Context manager can't close handle <_GIPCReader_49ff2e fd: 5>. It's locked for I/O operation out of context.
Traceback (most recent call last):
  File "ex3.py", line 29, in <module>
    main()
  File "ex3.py", line 12, in main
    assert pend.get() == "ACK"
  File "C:\Python27\lib\site-packages\gipc-0.7.0.dev0-py2.7.egg\gipc\gipc.py", line 862, in get
  File "C:\Python27\lib\site-packages\gipc-0.7.0.dev0-py2.7.egg\gipc\gipc.py", line 817, in _recv_in_buffer
EOFError: Most likely, the other pipe end is closed.

Curiously, on some other Windows servers (couldn't figure out if it's version specific or not :/) I'm getting a different error:

Administrator@host-ci124 (Windows 2016) ➜  ~ python ex3.py
Process _GProcess-1:
Traceback (most recent call last):
  File "C:\Cygwin\home\Administrator\python\lib\multiprocessing\process.py", line 258, in _bootstrap
    self.run()
  File "C:\Cygwin\home\Administrator\python\lib\multiprocessing\process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Cygwin\home\Administrator\python\lib\site-packages\gipc-0.7.0.dev0-py2.7.egg\gipc\gipc.py", line 365, in _child
    target(*args, **kwargs)
  File "C:\Cygwin\home\Administrator\ex3.py", line 25, in writer_process
    cend.put(time.time())
  File "C:\Cygwin\home\Administrator\python\lib\site-packages\gipc-0.7.0.dev0-py2.7.egg\gipc\gipc.py", line 962, in __exit__
    self[0].__exit__(exc_type, exc_value, traceback)
  File "C:\Cygwin\home\Administrator\python\lib\site-packages\gipc-0.7.0.dev0-py2.7.egg\gipc\gipc.py", line 772, in __exit__
    "operation out of context." % self))
GIPCLocked: Context manager can't close handle <_GIPCReader_ce02b5 fd: 5>. It's locked for I/O operation out of context.
Traceback (most recent call last):
  File "C:\Cygwin\home\Administrator\python\lib\site-packages\gevent-1.2.1-py2.7-win-amd64.egg\gevent\threadpool.py", line 211, in _worker
    value = func(*args, **kwargs)
OSError: [Errno 22] Invalid argument
Tue Sep 25 08:59:11 2018 (<ThreadPool at 0x16a1a90 0/1/10>, <built-in function write>) failed with OSError

Traceback (most recent call last):
  File "ex3.py", line 29, in <module>
    main()
  File "ex3.py", line 11, in main
    pend.put("SYN")
  File "C:\Cygwin\home\Administrator\python\lib\site-packages\gipc-0.7.0.dev0-py2.7.egg\gipc\gipc.py", line 931, in put
    self._write(struct.pack("!i", len(bindata)) + bindata)
  File "C:\Cygwin\home\Administrator\python\lib\site-packages\gipc-0.7.0.dev0-py2.7.egg\gipc\gipc.py", line 899, in _write
    bytes_written = _write_nonblocking(self._fd, bindata)
  File "C:\Cygwin\home\Administrator\python\lib\site-packages\gevent-1.2.1-py2.7-win-amd64.egg\gevent\os.py", line 141, in tp_write
    return get_hub().threadpool.apply(_write, (fd, buf))
  File "C:\Cygwin\home\Administrator\python\lib\site-packages\gevent-1.2.1-py2.7-win-amd64.egg\gevent\pool.py", line 326, in apply
    return self.spawn(func, *args, **kwds).get()
  File "C:\Cygwin\home\Administrator\python\lib\site-packages\gevent-1.2.1-py2.7-win-amd64.egg\gevent\event.py", line 385, in get
    return self.get(block=False)
  File "C:\Cygwin\home\Administrator\python\lib\site-packages\gevent-1.2.1-py2.7-win-amd64.egg\gevent\event.py", line 375, in get
    return self._raise_exception()
  File "C:\Cygwin\home\Administrator\python\lib\site-packages\gevent-1.2.1-py2.7-win-amd64.egg\gevent\event.py", line 355, in _raise_exception
    reraise(*self.exc_info)
  File "C:\Cygwin\home\Administrator\python\lib\site-packages\gevent-1.2.1-py2.7-win-amd64.egg\gevent\threadpool.py", line 211, in _worker
    value = func(*args, **kwargs)
OSError: [Errno 22] Invalid argument

It's worth noting that everything works with (ancient) gevent 1.0.2.

jgehrcke commented 6 years ago

Thank you for the feedback!

jgehrcke commented 6 years ago

I am pretty confident that this is now fixed in current master. Closing this for now.

If you want to try to test this please also use the most recent version of the code example: https://github.com/jgehrcke/gipc/blob/master/examples/synchronization.py