search

akiradeveloper / akashic-storage

Filey system backed S3-compatible storage in Scala/Akka-Http
Apache License 2.0
22 stars 2 forks source link

metadata caching #45

Closed akiradeveloper closed 8 years ago

akiradeveloper commented 8 years ago

akashic-storage saves various metadata such as acl, location, ... and they are files serialized by scala-pikcle library.

On each request, they are read from filesystem (hopefully it's cached in the backend) and deserialized or unpickled. They are really heavy because 1) IO goes out of JVM through the native operating system, so the run path is long 2) deserialization is, needless to say, heavy.

Fortunately, everything in akashic-storage is immutable and we can safely cache them.

One of my idea is to use guava caching but I don't believe it's scala-ish. I am looking for a nice library or a way to implement my own nicer one.

akiradeveloper commented 8 years ago

At the present my thought on this feature is negative because there is no way to safely cache object given that the distributed system is under the abstraction. If BasicFileAttributes#fileKey guarantees its uniqueness along the time it can be a good choice for the key for the lookup table but the reality is that it is implementation dependent.

I should keep akashic-storage versatile. So there is no solution until now.

akiradeveloper commented 8 years ago

implemented and documented