search

webgme / webgme-engine

WebGME server and Client API without a GUI
MIT License
11 stars 7 forks source link

Multiple simultaneous blob backends #172

Open brollb opened 4 years ago

brollb commented 4 years ago

Along with #171, enabling the blob server to support multiple backends simultaneously would allow webgme deployments to provide a "BYOS(torage)" like interface in which the users can opt to use their own existing storage resources such as SciServer's SciDrive or whatever resources their domain may already have (assuming it is supported by the deployment, of course).

kecso commented 4 years ago

We need to specify what you mean by these additional storages. If they are somewhat sources of artifacts, then they are not really alternative for blob storage. If we want alternatives to blob storage, they have to be structurally similar to our blob storage (you cannot really see or access a file path style containment structure). Also, many things are stored in the blob (every export) that is clearly not wanted by the user. So, although I see some use-case that would be interesting, we need to specify in detail what we really like here. Also, we are currently working on splitting the blob storage and allowing a temporary storage (not to counter this proposal, just saying that it might make more sense to think about where we want to allow more storages or user-based storages)

brollb commented 4 years ago

Yeah, I wanted to open an issue to start a discussion :)

Basically, in deepforge we are looking at creating a public deployment where people can attach their own compute and storage. This is especially desirable since many of the users will already have their own storage and compute resources. However, this means that the server will need to support different storage/compute adapters simultaneously. When I have been thinking about how to add support for multiple storage locations, I have been thinking that it could benefit other webgme deployments (which brought me to this issue).

Good point about the use of blob storage for many other tasks like export. I will probably explore making the asset types more generic (or replacing them w/ something more generic) in deepforge so assets can be stored in different locations. It might be nice if something like this was still made possible in webgme itself rather than just in deepforge...