search

cloudfoundry-attic / bosh-notes

Collection of proposals for BOSH
Apache License 2.0
51 stars 23 forks source link

Multiple CPIs on a single Director #3

Closed cppforlife closed 7 years ago

allomov commented 9 years ago

Yey! I've always waited for this feature. We already work with multiple cloud deployments and it would be really great if BOSH will be able to do it out of the box.

rkoster commented 9 years ago

Would this also include multiple instances of the same CPI? A usecase for this could be a HA cloud foundry installation with multiple openstack installations in different datacenters (in the wild I have not yet encountered a lot of private OpenStack installations which use the availability zones properly).

allomov commented 9 years ago

@rkoster sure, but such solution leaves a lot open questions. Currently open source CF is not implied to be deployed to hybrid clouds. First of all because of CC and UAA should use consistent data. We tried to implement hybrid solution with MariaDB in datacenters in different areas of the world: it was pretty hard and the latency was huge. I wouldn't say that current CF deployment to hybrid cloud will be HA. Still the best configuration I see is to use separate CF deployment to each cloud.

But this feature has lots of advantages besides of CF deployment to hybrid cloud.

cppforlife commented 8 years ago

@rkoster I think that is definitely one of the more useful cases why someone would use multi-CPI Director. I had interesting conversations specifically about OpenStack's use of that. Imho having ability to instantiate CPIs of the same type in the same Director is much more elegant then pushing down that kind of logic into the CPI itself.

cppforlife commented 8 years ago

With recent changes to how we deploy CPIs (collocated with the Director) I could see us taking a more dynamic approach to configuring CPIs -- may be via cloud-config?

allomov commented 8 years ago

Yes, cloud-config is a good option here. It allows to enable storing credentials for different clouds with little code changes.

Still running BOSH outside of the private network, where Job VM are placed, requires sharing several BOSH components with newly created Job VMs:

In my test deployment I created VPN connection between MicroBOSH instance and private network in external cloud. The fact that I had only one IP address for NATS and DAV blobstore made it easy to update manually CPI config to make bosh-agent think of VPN server as MicroBOSH instance. Still in case of BOSH cluster the task of sharing this resources could be more complex.

What do you think about the problem of sharing this resources across different networks?

dpb587-pivotal commented 7 years ago

Closing - this discussion and notes were migrated into multi-cpi.md. Currently, shared components like blobstore/nats must be accessible from each network, although there's no technical need for entire networks to be fully routable between each other.