It seems as though default signal handling doesn't work out of the box with the data generation script because big-sleep also uses this to gracefully exit during training.
So basically,
for prompt in prompts:
imagine() # Handles graceful exit
if terminate:
exit() # This doesn't work because the top-level signal handler doesn't get called
This means that during each iteration imagine() thinks that the program has been terminated, but this message is not passed on to the script performing the loop.
It seems as though default signal handling doesn't work out of the box with the data generation script because
big-sleep
also uses this to gracefully exit during training.So basically,
This means that during each iteration
imagine()
thinks that the program has been terminated, but this message is not passed on to the script performing the loop.