Possible unhandled error from worker

arcelien / pba

Efficient Learning of Augmentation Policy Schedules

https://bair.berkeley.edu/blog/2019/06/07/data_aug/

Apache License 2.0

505 stars 86 forks source link

Possible unhandled error from worker #26

Open snownus opened 5 years ago

snownus commented 5 years ago

@arcelien I came across an issue when extending PBA to my own network structure.

The error is like this: ERROR worker.py:1612 -- Possible unhandled error from worker: ray_RayModel:restore_from_object() (pid=49849, host=XXX) UnreconstructableError: Object 553134abdaa61cb3ed9fa555fcee22eb01000000 is lost (either evicted or explicitly deleted) and cannot be reconstructed.

Will this affect the final performance?

arcelien commented 5 years ago

Hey, I unfortunately haven't seen that error before. Perhaps you can rerun the code to see if it's just an intermittent issue, or open an issue over at https://github.com/ray-project/ray/issues.