search

fog / fog-openstack-core

fog's core openstack behaviors without API and cloud provider specifics
Apache License 2.0
6 stars 8 forks source link

How should we approach CRUD within model and collection objects? #7

Open wchrisjohnson opened 10 years ago

wchrisjohnson commented 10 years ago

As we (@mwhagedorn @elight @krames) work to refactor/rebuild the openstack fog provider, we need to agree on how to create/update/save/destroy objects from a number of perspectives:

1) How to create a single object - create directly, init/save, combination of the two? what gets returned? How can we minimize network activity, i.e.: what's the best approach from a performance perspective? Will it be necessary to create and return the new object in a single request/response cycle, or two distinct operations?

2) How should collections work? Do they operate as cached local copies of a remote resource, or simply as a way to operate against the remote resource via get, list, etc. If the former, then how do you handle it when an object exists in the remote collection/resource, but the local cache isn't up to date when you query it?

3) Do we want a common interface to models and collections in terms of methods implemented? Do we went to build a generator, or at least have a template to use if that's not worth the trouble for the time being?

What else do we need to decide around this topic? I'll collect this into a wiki once we gain consensus around the preferred approach(es).

krames commented 10 years ago

Let me take a stab at answering your questions.

  1. Fog currently allows you to create a new local object using the new method. You can then call the save method to transfer the state of the object to the remote cloud. I have seen this used a lot in storage. Say if a user has an existing container they could choose to retrieve a reference to the container using the get method which would retrieve the container metadata along with the first 10,000 files or they could use new which would allow them to interact with remote container without retrieving the file list. I have also seen services implement a head method (I believe HP does this) which would just retrieve the container metadata without the files list.
  2. I have seen people cache collections in fog, although, it is done very inconsistently. If we do opt to do that, I can see users attempting to use the cached object and having to rescue exceptions to deal with cache discrepancies. The user would then have to call reload and then retry their operation.
  3. Yes. Right now, fog collections all implement a all, get, reload, new, and create methods. The models all implement attributes, save, destroy and reload. (I am probably forgetting some) I would like to continue to see this pattern exist so that we can get more commonality amongst providers and services.

As far as other topics:

elight commented 10 years ago

@krames covered 1 well.

2) Baking in caching is desirable but impacts more than just OSC. Sure, we could roll our own but that's the entire problem with Fog: everyone has been rolling their own solution inside a library that should be providing unifying abstractions.

Recommend waiting for summit.

3) :thumbsup: Absolutely.

wchrisjohnson commented 10 years ago

@krames and @elight - are your numbers correct above? Am thinking some of those refs are not correct...

elight commented 10 years ago

Mine are since I edited my comment!

krames commented 10 years ago

I don't think markdown liked my extra paragraph for 1. I removed the break and everything is matching up now.

mwhagedorn commented 10 years ago

@krames.. yeah head methods are supported in the HP provider so that you can get metadata explicitely. I followed this same pattern for the IOS binding.

Its probably wise to keep the model layer as stupid as possible here. We dont know what kind of crazyiness people may want to do, and from what I have seen, if people want crazy efficient network accces and all of that, they go to the network layer and skip the model layer. I know that the HP management console (and a couple other examples I have seen) took this approach.

I am wondering if the model layer should just address the 80% use case and leave the heavy loads/big result sets to people using the request layer directly, as this has a historical precedent. Do we really want to be all things to all people on this?

krames commented 10 years ago

@mwhagedorn I would say we should stick with the element of least surprise. That being said...what is the element of least surprise? I think that might mean no caching. Maybe this is a question for our summit.

wchrisjohnson commented 10 years ago

I can definitely see where caching would make sense when it comes to storage. Other than that, unless you have a HUGE cloud infrastructure, I would say that caching is debatable, at best, as a feature.

elight commented 10 years ago

+1s all around. Yes, seriously.

On March 18, 2014 at 11:53:15 AM, Kyle Rames (notifications@github.com) wrote:

@mwhagedorn I would say we should stick with the element of least surprise. That being said...what is the element of least surprise? I think that might mean no caching. Maybe this is a question for our summit.

— Reply to this email directly or view it on GitHub.

mwhagedorn commented 10 years ago

If I had to guess polp would be no caching. For my swift ios app cache was done at the app layer

Sent from my iPhone

On Mar 18, 2014, at 11:39 AM, Evan Light notifications@github.com wrote:

+1s all around. Yes, seriously.

On March 18, 2014 at 11:53:15 AM, Kyle Rames (notifications@github.com) wrote:

@mwhagedorn I would say we should stick with the element of least surprise. That being said...what is the element of least surprise? I think that might mean no caching. Maybe this is a question for our summit.

— Reply to this email directly or view it on GitHub. — Reply to this email directly or view it on GitHub.