Even before https://github.com/pfnet/pfio/pull/297 , sparse file cache has been blown out by fork and thus sparse file caches created by each forked process duplicated inside single reader process(es). Thus, creating sparse file cache object before fork(2) let us multiple cache files. For example, if we use DataLoader (PyTorch) with 8 workers we have 9 sparse file cache instances on each child procs, where we intended single cache file being read by all data loaders.
This is clearly a severe bug in sparse file cache feature since it's initial experimental release https://github.com/pfnet/pfio/pull/275 - from 2.3.0 to 2.6.0,
Even before https://github.com/pfnet/pfio/pull/297 , sparse file cache has been blown out by fork and thus sparse file caches created by each forked process duplicated inside single reader process(es). Thus, creating sparse file cache object before fork(2) let us multiple cache files. For example, if we use DataLoader (PyTorch) with 8 workers we have 9 sparse file cache instances on each child procs, where we intended single cache file being read by all data loaders.
This is clearly a severe bug in sparse file cache feature since it's initial experimental release https://github.com/pfnet/pfio/pull/275 - from 2.3.0 to 2.6.0,