on the fly write gets affected with further conditions

[vitesh13]

When following code is implemented, it generates the perfect output record:

cmd = "DAMASK_grid -l tensionX.yaml -g 20grains16x16x16.vtr"
with open('check.txt','wb') as f:
  P = subprocess.Popen(cmd,stdout = subprocess.PIPE, stderr = subprocess.PIPE,shell=True)
  while P.poll() is None:
      f.write(line)
      #r = re.compile(' increment 3 converged')
     #for line in iter(P.stdout.readline, b''):
      # if re.search(r,P.stdout.readline().decode('utf-8')):
       #  print("hello")
         #os.kill(P.pid+1,signal.SIGUSR1)

But once we start checking for stdout, somehow there is a lag and the output record is not complete. Some outputs are missed out

cmd = "DAMASK_grid -l tensionX.yaml -g 20grains16x16x16.vtr"
#f = open('check.txt','w')

with open('check.txt','wb') as f:
  P = subprocess.Popen(cmd,stdout = subprocess.PIPE, stderr = subprocess.PIPE,shell=True)

  while P.poll() is None:
    r = re.compile(' increment 3 converged')
    for line in iter(P.stdout.readline, b''):
      if re.search(r,P.stdout.readline().decode('utf-8')):
        print("hello")
        os.kill(P.pid+1,signal.SIGUSR1)
      f.write(line)

[vitesh13]

This could be one of the ways to get the output recorded and send signal as well without loosing track of output:

cmd = "DAMASK_grid -l tensionX.yaml -g 20grains16x16x16.vtr"
with open('check.txt','wb') as f:
    P = subprocess.Popen(cmd,stdout = subprocess.PIPE, stderr = subprocess.PIPE,shell=True)
    r = re.compile(' increment 3 converged')
    record = []
    while P.poll() is None:
    for count,line in enumerate(iter(P.stdout.readline, b'')):
             record.append(line.decode('utf-8'))
             if count == 300:
                os.kill(P.pid+1,signal.SIGUSR1)

For realistic cases, the if statement should have something more realistic. And then write out the output separately (the only question would be what is the maximum size limit to lists? relevant for big simulations):

with open('check.txt','w') as f:
        for i in range(len(record)):
            f.write(record[i])

This recorded output gets the signal output as well