After looking through the code I found it very odd that items are put into queue[1] in run_workers() but run_workers() is not invoked in run(), therefore when train_loop() tries to get the predicted item from queue[1] it will be stuck indefinitely. I don't understand why is it written that way and not sure how to fix it. May I know if is this design is intended and are you able to run launch_speedup.py without any issues? Thanks.
Hi, I tried launching the
launch_speedup.py
but was not able to run any training due to the torch.multiprocessing queue intrain_loop()
being stuck. https://github.com/alexzhou907/DreamPropeller/blob/d7844af16edab4dfa94e124f2e3bdcf3a84268c9/launch_speedup.py#L298After looking through the code I found it very odd that items are put into
queue[1]
inrun_workers()
butrun_workers()
is not invoked inrun()
, therefore whentrain_loop()
tries to get the predicted item fromqueue[1]
it will be stuck indefinitely. I don't understand why is it written that way and not sure how to fix it. May I know if is this design is intended and are you able to runlaunch_speedup.py
without any issues? Thanks.