search

syndicate-storage / syndicate

Internet-scale software-defined storage system
Apache License 2.0
56 stars 10 forks source link

Named pipes for size-unbound files #117

Closed jcnelson closed 8 years ago

jcnelson commented 9 years ago

Often times, the AG will not know the size of a file it will serve. For example, the "file" could really be a datastream, or could be the result of a long-running process. Right now, we just say that the "file" has UINT64_MAX bytes if the size is not known.

A better abstraction would be to instantiate such files as named pipes. Wathsala and I tried this before in the UG, but we didn't realize that the UG will additionally need to connect to the named pipe as the writer process, despite also serving as the underlying filesystem.

I don't know if this approach will work--I will need to try it. I will close as "wontfix" if it's not possible.

jcnelson commented 8 years ago

WONTFIX. Not really a problem, and "pipe" isn't the right abstraction here since there can be multiple readers that expect the same data.