Add a generic "rackspace" provider

[russellhaering]

Rackspace now has a single global identity system, obviating the need for separate "rackspace-us" and "rackspace-uk" providers. I suggest adding a global "rackspace" provider, and eventually removing the existing ones.

[mattfarina]

Is there any architecture for handling providers? Right now I don't see an abstraction for providers but instead see it coded specifically to Rackspace.

[jamiehannaford]

@mattfarina +1. You could utilize a similar abstraction layer as Fog, which supports a plethora of providers. Obviously it'd have to cater to Go's unique sense of structs and interfaces, but it could probably be done.

[mattfarina]

Before this can really be called a multi-cloud provider it needs to have an abstraction layer that supports multiple cloud providers. Even just OpenStack ones other than Rackspace.

[sam-falvo]

When you authenticate, you specify a "provider" string. That string is used to index into a map to find a Provider structure. Everything is defined in https://github.com/rackspace/gophercloud/blob/master/global_context.go .

I'll be the first to admit this hardly an ideal solution, but it seemed like a good idea at the time, especially since providers remain relatively static over time. I was hesitant about exposing the Provider structure as a first-class API artifact, because I was concerned that its definition would change frequently. That didn't happen. So much for predicting the future!

I'm inclined to promote Provider as a first-class API artifact at this point, considering it's been very stable so far, and making the current set of module functions methods of the Provider struct. What are your thoughts on this change?

[mattfarina]

@sam-falvo The provider doesn't appear to provide enough extension for providers other than Rackspace. For example, In authenticate.go you have:

type User struct {
    Id, Name string
    XRaxDefaultRegion string `json:"RAX-AUTH:defaultRegion"`
    Roles []Role
}

That is Rackspace specific implementation details. A non-Rackspace OpenStack vendor or something other than OpenStack can't use the provider architecture to be part of this package.

[sam-falvo]

It's true that that field is part of a Rackspace-provided extension (remember: Gophercloud inherited code from Gorax), but the presence of this field does not preclude it from working with other OpenStack vendors. The field will be set to "" if unsupported, which is to be expected.

It seems you are looking for an example of how to authenticate against a non-Rackspace provider. This is entirely possible to do. We would love to have you contribute an example to Gophercloud that shows the world how to do this. We would be delighted to help review your code. Whether this be a simple design document or proposal, or full-on pull requests, we welcome your positive contributions to the project.

[mattfarina]

@sam-falvo what I'm looking for is an architecture that can handle OpenStack and other provider extensions. For example, both Rackspace and HP Cloud implement CDN differently. Where is an architecture capable of handling all use cases?

[jamiehannaford]

@mattfarina @sam-falvo I think a multi-provider implementation of Gophercloud would be much better than a single-provider (i.e. just OpenStack). Although Sam has highlighted that, for now at least, you could authenticate multiple providers by adjusting a field, the fact is that as the library grows to add more complex functionality, adjusting a value in the User struct won't really do much...

I think the key is to separate functionality whilst at the same time abstracting core features for re-use. Many existing SDKs already do this: Fog, for example, has multiple providers (Rackspace, Google, AWS); and each provider has different services (compute, object storage, block storage). The main sticking factor we have to get over with Gophercloud is that we're not dealing with an OOP language that can be easily abstracted.

A possible file structure could be:

core
   authenticate.go
   compute.go
   object.go

rackspace
   authenticate.go
   compute.go
   object.go

rackspace.go

I think anonymous fields in structs are going to be crucial, too:

type CoreCompute struct {
   id int
}

func (this CoreCompute) reboot() {
   // core create
}

func (this CoreCompute) delete() {
   // core delete
}

type RackspaceCompute struct {
   CoreCompute
}

func (this RackspaceCompute) delete() {
   // custom Rackspace delete
}

func main() {
   compute := RackspaceCompute{CoreCompute{12345}}

   // "inherits" from core
   compute.reboot()

   // rackspace delete
   compute.delete()
}

This is just a basic idea. We could also use interfaces to make sure structs which act as models abide by a fixed behaviour:

type PersistentThinger interface {
   Create()
   Update()
   Delete()
}

type Computer interface {
   Reboot()
}

Using interfaces and anonymous fields, we could easily implement multiple inheritance.

[mattfarina]

I'm waiting to see how this provider architecture comes together.

Also note, Rackspace has extensions to OpenStack. HP Cloud has extensions to OpenStack that are different from Rackspace. So, for providers there is OpenStack (vanilla), Rackspace, HP Cloud, and others. This is just in the OpenStack space. Can't assume that what works for Rackspace works for all of OpenStack users.

[jamiehannaford]

@mattfarina You'd leave the implementation of unique functionality up to the individual Provider. So if HP Cloud is completely different from Rackspace, that's no biggie: they both might inherit shared functionality common to all providers (i.e. by inheriting a CommonProvider or a CommonService), but they'd generally dictate their own implementation.

[mattfarina]

@jamiehannaford what you really have is a general OpenStack provider, then Rackspace and HP Cloud ones that extend a general OpenStack one. Calling a Rackspace provider OpenStack isn't really true. It's OpenStack plus one vendors extensions.

I'd just like to see the architecture that accommodates all of this plus alternate providers like AWS and Azure. That's what a multi-cloud library should be doing. What's missing is the abstractions.

[jamiehannaford]

@mattfarina Correct. You'd have a general OpenStack Provider that can be used independently, and that can also be used as an abstraction for other Providers to inherit and override (i.e. Rackspace). Other Providers like AWS or Google would not inherit this OpenStack layer.

This library is very much still in an early stage of development (version 0.0.0); so changes -- whether code-based or architectural -- will be integrated into the SDK as time goes by. In my personal opinion based on what I've experienced so far with Go, it's hard to model a client library the same way as traditional OO bindings. It's very different and probably requires a different approach, which takes a bit of time.

[sam-falvo]

@mattfarina Constructive criticism is appreciated, but code and design contributions provide more value. You would have a greater impact if you isolated specific issues, illustrate why they're problems, but most importantly, provide potential solutions as well.

[sam-falvo]

Case in point, I've spent the majority of my time this week reviewing the Cloud Files API, and thinking hard about how best to expose this interface using Go's standard Reader, Writer, Seeker, et. al. set of interfaces. It's not a trivial mapping. OpenStack has basically replicated S3's semantics (if I understand the docs rightly), which throws us more or less back to the days of datasets (as on IBM mainframes) and the use of "access methods" to work with them. Meanwhile, Go's file abstraction map more naturally to Unix and Plan 9 file and directory semantics.

This API will be particularly volatile as we learn more about the intricacies of the Cloud Files API. The abstraction will evolve over time.

[jnoller]

The abstractions will come as provider support is fleshed out as Jamie says: you're perfectly right that rackspace is OpenStack += some extensions and the abstractions we create should support that, just like fog, jclouds, libcloud and others

On Aug 14, 2013, at 1:30 PM, Matt Farina notifications@github.com wrote:

@jamiehannaford what you really have is a general OpenStack provider, then Rackspace and HP Cloud ones that extend a general OpenStack one. Calling a Rackspace provider OpenStack isn't really true. It's OpenStack plus one vendors extensions.

I'd just like to see the architecture that accommodates all of this plus alternate providers like AWS and Azure. That's what a multi-cloud library should be doing. What's missing is the abstractions.

— Reply to this email directly or view it on GitHub.

[sam-falvo]

Getting back on the topic of this ticket, it works as intended; users in UK should use their the LON address, while users in the US should use the US address. Otherwise those two endpoints should behave identically, per global auth features.

To continue the other discussion, I invite participants to join the gophercloud-dev mailing list. See http://gophercloud.io