Open up internal ports within Cloud Foundry cluster

[anderssv]

I really think this might be a BOSH issue, but posting here hoping some of you guys have come across this with this setup.

Every thing is working, but I decide to expand so I do bosh cf change deas 1 . bosh deploy completes and an extra node exists. But vmc push fails on the "checking" step.

I see from the logs on cc that it's accessing the rails stuff there, but I really don't know where to start looking to see if it is working with the DEA. Any hints?

[drnic]

Hmm, there's a part of me that cannot confirm that I tested this working. So it's perhaps not a regression but an incomplete feature.

Dr Nic Williams Stark & Wayne LLC - consultancy for Cloud Foundry users http://drnicwilliams.com http://starkandwayne.com cell +1 (415) 860-2185 twitter @drnic

[anderssv]

It didn't fix everything, but I found this in the logs on the DEA:

[2013-02-20 15:30:40.675723] dea - pid=31736 tid=49e6 fid=4f7f   WARN -- Failed to download app bits from http://10.33.204.170:9022/staged_droplets/9/db096c300635377575488fbb541d8202a719d6e2
[2013-02-20 15:30:40.708730] dea - pid=31736 tid=49e6 fid=48f2   WARN -- Failed unzipping droplet, command exited with status pid 32079 exit 2

After opening the port in the firewall rules, it got through. But still having problems accessing the real application.

Any way to automate the firewall change?

[anderssv]

Added a general rule for the Security Group to allow all traffic between nodes in the cloudfoundry-production group and it worked. Awesome. :)

[anderssv]

So I guess the basics here is the 9022 port from DEA to controller, and controller needs access to the ports that apps will be hosted on the DEA. Don't know if that's a certain range. I opened 1024-65535.

[drnic]

The VMs should all be able to talk to one another without opening public ports. I think there is something we can add to the cloudfoundry-NAME security group for this.

Dr Nic Williams Stark & Wayne LLC - consultancy for Cloud Foundry users http://drnicwilliams.com http://starkandwayne.com cell +1 (415) 860-2185 twitter @drnic

[anderssv]

Yeah, I didn't open it to everyone. I set the source to be the same security group. Seems you need to know the id (it helped me find it in auto-complete), but setting that as source for all those ports worked.

[drnic]

Ok, can you please re-title this Issue to "Open up internal ports within Cloud Foundry cluster" and list the ports & port ranges you think we need to open. Yes we can open all of them, but perhaps the fewer the better. I'm no security nut; but we are letting external people run their own code on our DEA nodes... :)

We also have the ability to create multiple security groups and assign them to VMs, e.g. a "cloudfoundry-dea" group, which could open a wide range of ports; and "cloudfoundry-cc" could open 9022.

[anderssv]

I'll see if I can see some patterns tomorrow. The CF guys should be able to tell if there's a specific range they map to the deployed applications for access through the proxy.

[mrdavidlaing]

And, as I discovered, having 2 security groups that both open the same portis no problem.

On Wednesday, February 20, 2013, Dr Nic Williams wrote:

Ok, can you please re-title this Issue to "Open up internal ports within Cloud Foundry cluster" and list the ports & port ranges you think we need to open. Yes we can open all of them, but perhaps the fewer the better. I'm no security nut; but we are letting external people run their own code on our DEA nodes... :)

We also have the ability to create multiple security groups and assign them to VMs, e.g. a "cloudfoundry-dea" group, which could open a wide range of ports; and "cloudfoundry-cc" could open 9022.

— Reply to this email directly or view it on GitHubhttps://github.com/StarkAndWayne/bosh-cloudfoundry/issues/112#issuecomment-13841974.

David Laing Open source @ City Index - github.com/cityindex http://davidlaing.com Twitter: @davidlaing

[anderssv]

Just did a quick check now for the ports that is opened for handling application instances and need to be open for the router. Did not see any real patterns for it. Seems to be from the low thousands and upwards.