[solved] cloud-config instructions not being executed, system coming back with node unreachable

asimkhawaja commented 8 years ago

Hey Folks, I am unable to use cloud-config in my fittings file. Running with the -d setting shows that when plumbery tries to use cloud-config it get a node unreachable. I have based my simplified fittings file based on the more complex files in the samples project, so I might be missing something.

Your guidance would be most appreciated.

Fittings file used:

information:
    - "Setup mysql servers for testing on Ubuntu /Redhat "
links:
  documentation: https://plumbery.readthedocs.org/en/latest/tutorial.apache2.cluster.html

defaults:

  cloud-config:

    ssh_keys:
      rsa_private: |
        {{ key.rsa_private }}
      rsa_public: "{{ key.rsa_public }}"

    users:
      - default

      - name: ubuntu
        sudo: 'ALL=(ALL) NOPASSWD:ALL'

    disable_root: false
    ssh_pwauth: true

---

# London
locationId: AU10
regionId: dd-au

blueprints:

  - mysql:

      domain:
        name: BaaS_Test_AU10

        service: advanced

      ethernet:
        name: "baas_vlan"
        subnet: 10.0.0.0

      nodes:
        - mysqltesti_2:
           information : "mysql base node for ubuntu"
           appliance : 'CentOS 7'
           cloud-config:
              packages:
               - ntp
               - mysql-server

plumbery output:

[root@localhost Development]# python -d -m plumbery asim_fittings.yaml prepare
Using polisher 'prepare'
Polishing all blueprints
Plumbing at 'AU10' Australia - Melbourne MCP2 (Australia)
preparing node 'mysqltesti_2'
- not found
Worked for you in a blink locally, and for 1 seconds in the cloud
[root@localhost Development]# python -d -m plumbery asim_fittings.yaml deploy
Building all blueprints
Plumbing at 'AU10' Australia - Melbourne MCP2 (Australia)
Creating network domain 'BaaS_Test_AU10'
- already there
Creating Ethernet network 'baas_vlan'
- already there
Creating node 'mysqltesti_2'
- in progress
Using polisher 'configure'
Polishing all blueprints
Plumbing at 'AU10' Australia - Melbourne MCP2 (Australia)
Configuring blueprint 'mysql'
- waiting for nodes to be deployed
- nodes have been deployed
Configuring node 'mysqltesti_2'
Starting nodes from all blueprints
Plumbing at 'AU10' Australia - Melbourne MCP2 (Australia)
Starting node 'mysqltesti_2'
- in progress
Using polisher 'prepare'
Polishing all blueprints
Plumbing at 'AU10' Australia - Melbourne MCP2 (Australia)
preparing node 'mysqltesti_2'
- using cloud-config
- node is unreachable
Using polisher 'information'
Polishing all blueprints
Plumbing at 'AU10' Australia - Melbourne MCP2 (Australia)
- examinating node 'mysqltesti_2'
Showing information
About this fittings plan:
- Setup mysql servers for testing on Ubuntu /Redhat
- documentation: https://plumbery.readthedocs.org/en/latest/tutorial.apache2.cluster.html
About 'mysqltesti_2':
- node is up and running
Worked for you for 6 seconds locally, and for 6 minutes in the cloud
[root@localhost Development]# python -d -m plumbery asim_fittings.yaml prepare
Using polisher 'prepare'
Polishing all blueprints
Plumbing at 'AU10' Australia - Melbourne MCP2 (Australia)
preparing node 'mysqltesti_2'
- using cloud-config
- node is unreachable
Worked for you in a blink locally, and for 2 seconds in the cloud

tonybaloney commented 8 years ago

Hi @asimkhawaja it's hard to read the logs. Can you edit the message to use the pre-formatted syntax in Markdown please? https://help.github.com/articles/working-with-advanced-formatting/

asimkhawaja commented 8 years ago

No worries. Is that better?

asimkhawaja commented 8 years ago

I think what the issue may be. I am running plumbery from my local machine agasint CaaS. I need to perhaps run it directly on the CaaS as per this comment in the polisher file:

 In the real life when you have to prepare any appliance, you need to be close
    to the stuff and to touch it. This is the same for virtual fittings.
    This polisher has the need to communicate directly with target
    nodes over the network.
    This connectivity can become quite complicated because of the potential mix
    of private and public networks, firewalls, etc. To stay safe plumbery
    enforces a simple beachheading model, where network connectivity with end
    nodes is a no brainer.
    This model is based on predefined network addresses for plumbery itself,
    as in the snippet below::
        ---
        # Frankfurt in Europe
        locationId: EU6
        regionId: dd-eu
        # network subnets are 10.1.x.y
        prepare:
          - beachhead: 10.1.3.4
    Here nodes at EU6 will be prepared only if the machine that is
    executing plumbery has the adress 10.1.3.4. In other cases, plumbery will
    state that the location is out of reach.

tonybaloney commented 8 years ago

You can run the docker container in CaaS or connect to the VPN

Sent from Outlook Mobile

On Mon, May 2, 2016 at 6:44 PM -0700, "asimkhawaja" notifications@github.com wrote:

I think what the issue may be. I am running plumbery from my local machine agasint CaaS. I need to perhaps run it directly on the CaaS as per this comment in the polisher file:

In the real life when you have to prepare any appliance, you need to be close

to the stuff and to touch it. This is the same for virtual fittings.

This polisher has the need to communicate directly with target

nodes over the network.

This connectivity can become quite complicated because of the potential mix

of private and public networks, firewalls, etc. To stay safe plumbery

enforces a simple beachheading model, where network connectivity with end

nodes is a no brainer.

This model is based on predefined network addresses for plumbery itself,

as in the snippet below::

    ---

    # Frankfurt in Europe

    locationId: EU6

    regionId: dd-eu

    # network subnets are 10.1.x.y

    prepare:

      - beachhead: 10.1.3.4

Here nodes at EU6 will be prepared only if the machine that is

executing plumbery has the adress 10.1.3.4. In other cases, plumbery will

state that the location is out of reach.

— You are receiving this because you commented. Reply to this email directly or view it on GitHub

asimkhawaja commented 8 years ago

I am now able to get to a stage where the beachead network is working when i run it from a VM in CaaS. But now it seems like cloud-init is just not executing the instructions on the machine. Will close and make a new ticket for this next problem

bernard357 commented 8 years ago

@asimkhawaja , some clarification coming lately. Cloud-config requires SSH connectivity between the node that is running plumbery and the target node. There are at least too common ways to ensure this:

run plumbery in a VM, and ask it to create nodes in the same network domain and on the same network
run plumbery anywhere, and add a "glue:" directive to your fittings plan to make node reachable over SSH via internet . Look for example at https://github.com/DimensionDataCBUSydney/plumbery-contrib/blob/master/fittings/example/first/fittings.yaml

This would deserve more comprehensive explanations in the on-line documentation, of course. Many thanks for your efforts on plumbery :-)

NTTLimitedRD / plumbery

[solved] cloud-config instructions not being executed, system coming back with node unreachable #25

Fittings file used:

plumbery output: