loose coupling with the mounted filesystem

[akiradeveloper]

Now akashic-storage can only be built on a mounted filesystem. So, it's possible to built on NFS or other distributed filesystems if they are capable of exposing filesystem to the operating systems.

Let's borrow the idea from nfs-ganesha's FSAL. It's defined by nfs-ganesha and filesystem providers are only responsible to implement their own FSAL (like FSAL_GLUSTER). This way, nfs-ganesha bypasses operating system paths or thick fuse layer.

Just like nfs-ganesha, I would like to define my own filesystem-like interface but also looking for well-defined existing interface of such kind. Any ideas?

[akiradeveloper]

Maybe it's quite a simple set required:

case class Attr(creationDate: Date, length: Long)
trait FileSystemLike {
  type Node
  def getRoot: Node
  def makeDir(Node, name)
  def remove(Node)
  def create(Node, name, Array[Byte]): Node
  def listDir(Node): [Node]
  def getInputStream: InputStream
  def getSource: Source[ByteString, Future[Long]] =
     StreamConverters.fromInputStream(getInputStream, chunkSize)
  getAttr(Node): Attr
}
// example
class Local extends FileSystemLike {
  type Node = Path
  def getRoot: Node
  ...
}

and a QuickTest for FileSystemLike

[akiradeveloper]

config -> (FileSystemLike, config) Server(FileSystemLike, config)

config = akashic.storage { backend-type = Default ... ... }

Default.decompose(config): (FileSystemLike, config)

[akiradeveloper]

getFile could be more stream-like

  def fromFile(f: File, chunkSize: Int = 8192): Source[ByteString, Future[Long]] =
    new Source(new FileSource(f, chunkSize, DefaultAttributes.fileSource, sourceShape("FileSource")))

              complete {
                HttpEntity.Default(contentType, file.length,
                  FileIO.fromFile(file).withAttributes(ActorAttributes.dispatcher(settings.fileIODispatcher)))
              }

If we can get Source[ByteString, Future[Long]] then we win

[akiradeveloper]

Oh yeah

Use StreamConverters.fromInputStream(() => InputStream, chunkSize): Source[ByteString, Future[Long]]

InputStream -> Source[ByteString] -> HttpEntity.Default -> completion

[akiradeveloper]

If you like this plan, please vote and I will be quicker

[akiradeveloper]

Oh no... I must do this because it changes the conf

[akiradeveloper]

started on filesystemlike branch

[akiradeveloper]

done (https://github.com/akiradeveloper/akashic-storage/pull/71)

[akiradeveloper]

Maybe, it's better to use Stream[Chunk] as the input/output to/from a file. That way, I need to implement Stream[Chunk] -> InputStream but it's easy