akiradeveloper / akashic-storage

Filey system backed S3-compatible storage in Scala/Akka-Http
Apache License 2.0
22 stars 2 forks source link

Make metadata caching less strict #118

Closed akiradeveloper closed 8 years ago

akiradeveloper commented 8 years ago

Now the cache key must be defined to use metadata caching (Cached trait), but it's bit difficult to understand the correctness by the document.

If we can allow eventual consistency as AWS S3 does, querying if the data is changed every seconds and then change the in-memory cache could be allowable.

akiradeveloper commented 8 years ago

This solution can solve #117 too

akiradeveloper commented 8 years ago
$ git di
diff --git a/src/main/scala/akashic/storage/admin/UserDB.scala b/src/main/scala/akashic/storage/admin/UserDB.scala
index eb420f5..c0e133d 100644
--- a/src/main/scala/akashic/storage/admin/UserDB.scala
+++ b/src/main/scala/akashic/storage/admin/UserDB.scala
@@ -60,6 +60,7 @@ case class UserDB(root: NodePath) {
       userMap.values.toSeq.pickle.value
     }
     def commit {
+      Thread.sleep(1000)
       Commit.replaceData(dbData, makeCache) { data =>
         data.replace(this, Cache.creationTimeOf(dbData.filePath))
       }
diff --git a/src/main/scala/akashic/storage/caching/Cache.scala b/src/main/scala/akashic/storage/caching/Cache.scala
index 560a448..ce8704c 100644
--- a/src/main/scala/akashic/storage/caching/Cache.scala
+++ b/src/main/scala/akashic/storage/caching/Cache.scala
@@ -51,10 +51,10 @@ trait Cache[V] extends Data[V] {
     val bytes = writer(v)
     filePath.createFile(bytes)

-    while (filePath.getAttr.creationTime < creTime + 1000) {
-      filePath.removeIfExists
-      Thread.sleep(1000)
-      filePath.createFile(bytes)
-    }
+//    while (filePath.getAttr.creationTime < creTime + 1000) {
+//      filePath.removeIfExists
+//      Thread.sleep(1000)
+//      filePath.createFile(bytes)
+//    }
   }
 }

with this change, we can avoid waiting in overwrite. The side effect is 2 tests fail: overwrite and read after write. Because there are tests in s3-tests that expect overwriting an object, I decide to not to take this change but wait in immediate overwrite.