Closed y1r closed 1 year ago
/test
Successfully created a job for commit dffff08:
Successfully created a job for commit dffff08:
/test
Successfully created a job for commit e77521d:
Successfully created a job for commit e77521d:
/test
Successfully created a job for commit d407139:
Successfully created a job for commit d407139:
/test
Successfully created a job for commit 612ad31:
Successfully created a job for commit 612ad31:
https://github.com/pfnet/pfio/pull/292/commits/612ad312c01199f336fbbb115072b9fc48483d10 fixes the test.
@kuenishi Please review this PR.
Current PFIO doesn't support S3 + zipfile on fork-server based multiprocessing execution model. This model requires parameters given to Process should be pickled to move object from child to parent, who has responsibility to fork an new process.
https://github.com/pfnet/pfio/pull/292/commits/dffff08c83c3fb52147ab53029d212625828a1c3 solves this issue by avoiding pickling of fileobj and zipobj, then after fork, it will be re-created by zip and S3's _reset() functions.