Pythons 3.8.2 subprocess.poll( ) hangs after being called 40 or 50 times

KeithProctor commented 3 years ago

I just upgraded to MacOS 11.2 which now uses Python 3.8.2. My test suite guarantees that printed output prints as the output is sent to the console by using the code snippet below. As verified by the print xxx or print yyy statements the code hangs after roughly 50 Unit Tests. This code has been working for roughly 4 years and of course was working in the Python that ships with Big Sur 11.1. Any ideas for work arounds would be appreciated. My code currently runs on Mac, Windows and Centos.

print( '\nrunning: ' + curFilePath.split('/')[-1] + ': ' + str(curVer))
p = subprocess.Popen(fullCMD, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)

while True:     # Print output as it sent to console
    print( 'xxx' )
    nextline = p.stdout.readline().decode()
    if nextline == '' and p.poll() is not None:
        print( 'yyy' )
        break
    sys.stdout.write(nextline)
    sys.stdout.flush()

print( 'completed: ' + curFilePath.split('/')[-1] )

This could possibly be the call to decode. First time I got a stack dump from poll() but the last few times I've been getting this when I Control-C the application.

Unit Test Class: TestCase
^CTraceback (most recent call last):
  File "dapiRunner.py", line 93, in <module>
    nextline = p.stdout.readline().decode()
KeyboardInterrupt

KeithProctor commented 3 years ago

I have tried using print with flush set to true. Could there possibly be other work arounds?

KeithProctor commented 3 years ago

I tried this as well... which also fails.

import io
for line in io.TextIOWrapper(p.stdout, encoding='utf-8'):
    print( line )
    p.stdout.flush()

matrixise commented 3 years ago

@ewdurbin Could you close this one? Thanks

KeithProctor commented 3 years ago

Sorry, Stéphane, There wasn't a link to the issue and searching doesn't bring it up.

 Keith Proctor @.*** Software Development Engineer Claris International Inc., formerly FileMaker, Inc. M: 01-408-987-7461

This email and any attachments may be privileged and may contain confidential information intended only for the recipient(s) named above. Any other distribution, forwarding, copying or disclosure of this message is strictly prohibited. If you have received this email in error, please notify me immediately by telephone or return email, and delete this message from your system.

On Apr 26, 2021, at 1:37 PM, Stéphane Wirtel @.***> wrote:

@ewdurbin https://github.com/ewdurbin Could you close this one? Thanks

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/python/psf-infra-meta/issues/97#issuecomment-827130728, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGRQZBSENYRUWYIIIPHXG3DTKXFH7ANCNFSM4XDTEPSA.

python / psf-infra-meta

Pythons 3.8.2 subprocess.poll( ) hangs after being called 40 or 50 times #97