I set up a read-only cache with lru strategy, which should not have dirty data problems.
Now about 10 days past, I went into all kinds of data corruption. My ipynb codes can't be read. Data frequently encounter load error. My trained models are not readable.
I've never had these kinds of problems before. Even though the small files frequently bump into error, I always assumed that I saved them wrongly or closed unproperly. Until I just found out my trained models are not readable, while just yesterday I have used them for test.
Most of the corrupted files would give no hint, the loading methods report corruption. While some of them will get info like this:
du: cannot access 'E10036569.npy': Structure needs cleaning
I don't know how to reproduce the problem.
I use ubuntu 18.04 and a samsung ssd as cache for a raid5.
Now I have deleted the cache device. Just hope that anyone who noticed this thread, will be very cautious to use this project.
I set up a read-only cache with lru strategy, which should not have dirty data problems. Now about 10 days past, I went into all kinds of data corruption. My ipynb codes can't be read. Data frequently encounter load error. My trained models are not readable. I've never had these kinds of problems before. Even though the small files frequently bump into error, I always assumed that I saved them wrongly or closed unproperly. Until I just found out my trained models are not readable, while just yesterday I have used them for test.
Most of the corrupted files would give no hint, the loading methods report corruption. While some of them will get info like this:
du: cannot access 'E10036569.npy': Structure needs cleaningI don't know how to reproduce the problem. I use ubuntu 18.04 and a samsung ssd as cache for a raid5.
Now I have deleted the cache device. Just hope that anyone who noticed this thread, will be very cautious to use this project.