Matorage is tensor(multidimensional matrix) object storage manager for deep learning framework(Pytorch, Tensorflow V2, Keras)
73
stars
8
forks
source link
When there are many cores during multi-processing, the metadata and the actually uploaded file are not synced. #36
Open
graykode opened 3 years ago
I will write test code related to this and conduct experiments.
To implement it correctly, you should check https://github.com/graykode/matorage/blob/master/matorage/uploader.py#L41 to see if the upload is a success(200) code and put it in metadata.json only when it succeeds.