Storage.get() should accept 'preferred' instance

[grabbou]

For example:

Storage.get(['amazon', 'localsystem']).upload(...) Storage.get([Storage.Providers.AmazonS3]).upload(...)

will try to return first available storage. If none of above is available, will return default one.

Note: if specified as a string -> we look for a name. If passed as an object -> we try to return first instance of a given type available.

[grabbou]

@JedWatson @JohnnyEstilles ?

[estilles]

will try to return first available storage. If none of above is available, will return default one.

Can you explain cases when they would not be available? Do you mean if they're not configured? Or ... an error was returned upon attempting to upload?

I guess I'm just trying to figure out the use cases for this. Maybe I just need my morning coffee. :-)

[grabbou]

@JohnnyEstilles, actually, after thinking about that for a while -> I can't :dancer: People assume that get is a kind of strict method. If we can't get what we want - we should stop.

So we just might want to extend get method to accept:

• nothing (returns default, we are happy with everything)
• string/object (returns it!)
• array of strings/objects (resolves using above method - recursively). Useful when we might be satisfied with Amazon / Azure only (for example when we don't want to store Images in MongoDB/LocalSystem under any circumstances)

Not available provider -> neither 'string' or type of the provider are configured (user hasn't passed them via add or init method)

In the future, if somebody requests that feature, we might add getPreferred method to act exactly like get but without stopping when specified providers are not available. Instead of that - will return default one. Or -> we can add another param to get(), Boolean nonStrict. Behaviour the same.

[estilles]

Hmm, I understand we're trying to be as flexible as possible. However, I thought the intention was to let the Keystone developer choose what ever storage provider he/she desires (i.e. if I want to store an image, as a developer, I can decide whether I want to store it in Amazon or Azure or the LocalSystem). I can understand a field component developer "suggesting" what he/she considers the best provider for his component, but that would (and should) be just a suggestion.

In my opinion, Storage.get('string') is a kind of strict method, Storage.get('amazon') should return an error when Amazon is not configured.

In this use case, shouldn't the Keystone developer be the one specifying which storage provider to use? And if that's the case he/she should have configured that provider.

If, on the other hand, the component developer is restricting the providers he/she will allow for the component, then that simply narrows the number of providers available to the Keystone developer. It is still his/her responsibility to chose among the available alternatives.

I thought maybe the idea of specifying multiple providers was maybe to use a s fallback. Like my first choice is not available (e.g. it's unreachable, it's full, or returns some other sort of error) then it would fall back to my second choice. That maybe I can understand. But I guess I fail to see any other use cases for this. Then again, maybe I'm just not looking at it from the right perspective. If that's the case please enlighten me. :-) (Currently on coffee cup number 2 ... so I may get it soon. :-)

[grabbou]

Yep, but @JohnnyEstilles, we have to separate Developers (us, Keystone authors) from Users (normal developers and end-users).

In this use case, shouldn't the Keystone developer be the one specifying which storage provider to use? And if that's the case he/she should have configured that provider.

Users are the ones that specify their providers (because everybody has different accounts and paid plans). Our plugins written by developers should be as flexible as possible. That's why we can call Storage.get() to get anything available or Storage.get('String') to pass a name given to us by our users.

In our use-case, in Keystone, we will be simply getting instance of a provider passed to a Field constructor by the user. So - if he will decide to use 'AmazonS3' as a Field provider, we are going to call Storage.get(this._provider) inside a Field class behind the scenes!

The rest of the options is intended to be used in other projects (we are trying to be as generic as possible) or in other Keystone parts (for example when Users are writing their own views or forms with uploaders).

I might be writing another, standalone plugin, for image upload (powered by Node, let's say). I want to depend on that library but I want to restrict storage providers only to cloud ones (array). Although I expect that this option is not going to be very popular (because nobody wants to made a list of available providers smaller then it is in fact), it's only one-line of code that might be helpful for somebody in the future.

[webteckie]

@grabbou assuming I understand the concern here are my $.02 cents:

Considering keystone, I would think that only DB storage (the same DB keystone is connected to) would override the keystone configured file system. To keep it simple, I would think a field would have the capability to stream its content to either the configured storage system or to the DB. So I can envision something like the following setup:

keystone init( "storage system": [azure | s3 | DB | ... | or can also be defined by some env variable] ) *\ and this can be different per environment (DEV / PROD)

keystone fields: { Type.SOME-FIELD-THAT-REQUIRES-STORAGE dbstorage: [true | false] }

Thus, setting dbstorage to true would imply the field prefers to stream the contents to DB vs. the configured storage system (assuming the storage system is not also DB). This seems simple since the field can just check the dbstorage flag and stream the content appropriately. I think the assumption would be that keystone will always activate the DB streaming interface regardless or not it was configured as the storage system. I, for example, "storage system" was set to azure then behind the scenes keystone would activate the DB stream. Of course, if the user just configures "storage system" as DB then the setting of dbstorage is ignored.

[estilles]

I understand what you're saying. I guess we're just using different lingo. When I say "Keystone Developers" I was referring to developer's who use Keystone (which you call Keystone users). I was calling "component developers" those who will develop components for Keystone, regardless of whether those components will be part of the Keystone core or not.

So we're basically saying exactly the same thing.

In our use-case, in Keystone, we will be simply getting instance of a provider passed to a Field constructor by the user. So - if he will decide to use 'AmazonS3' as a Field provider, we are going to call Storage.get(this._provider) inside a Field class behind the scenes!

If a component developer restricts the storage provider to Amazon, then the Keystone developers (or as you call them users) are required to use that provider, hence they should have configured that provider. Correct?

[grabbou]

Another interesting thing to mention (before I go to sleep) is that:

For now, we simply throw an error if provider instance fails to init (can't ensure container or credentials provided are falsy). Instead of that, we might simply deregister failed provider and continue.

Then, in our code, if we initialise User model with field Avatar that uses [LocalStorage, AmazonS3] for storing images, we can automatically switch to AmazonS3 in our prod env if we are deploying to Heroku (instead of stopping application in Heroku, we switch).

But, on the other hand, if one is aware of that, he should use AmazonS3 by default just to avoid errors and weird behaviours.

On the another hand..... he can configure init() array keeping appropriate order and get() (without any params) will return the first one available.

@JohnnyEstilles - yep, correct / not XD Actually -> in Keystone, users can specify whatever they want to use (unless we write additional checks in our Fields). I think that use-case above is the most accurate one.

[estilles]

That is an interesting point. Cool debate! :-)