search

petersirka / node-filestorage

Storage for storing uploaded files in node.js
MIT License
72 stars 12 forks source link

[question] Does/Would filestoreage support clustering #3

Closed gnetsys closed 10 years ago

gnetsys commented 10 years ago

Does file storage support clustering? or Would it eventually support it?

petersirka commented 10 years ago

Clustering no ... Maybe for read - yes, but for write - no. It's not easy create app with cluster because apps must communicate between them (in most cases).

grantcarthew commented 9 years ago

Hi @petersirka ,

I was thinking about this issue of scalability for the future of my apps. I am using node-filestorage because I don't want to pay money for a cloud solution. If my apps ever takes off, my only bottle neck for scaling out is node-filestorage. I believe a volume mounted on a scalable file system like GlusterFS would work, as long as node-filestorage supported concurrent writes.

Couldn't lockfile be integrated into node-filestorage as a dependency and on all writes lock a common file then unlocking on success? This would prevent conflicts on write at a cost of slowing down performance when duplicate write operations are attempted at the same time. The lockfile option could be disabled by a simple config for single node installations.

What do you think?

Another option would be to store the config or node-filestorage database into a document database rather than on the file system. Give a config for the database access etc.

This is all adding complexity but giving cheap storage for small shops.

It may be easier to migrate to cloud or openstack swift solutions if I ever need to scale out.

I don't have the skills or the time to help yet, but it may be an interesting project to add scale to node-filestorage.

I do love the solution you have come up with Peter.

CC: @gnetsys