Project selection - Githubissues

wang542 commented 5 years ago

Regarding project selection of Yanting hid: fa19-516-170 and myself hid: fa19-516-157. As you may recall we both wanted to do the cloudmesh-ai with Qiwei hid: fa19-516-151. It is my understanding that we would need azure compute service as a base of the cloudmesh-ai service, for comparing purposes. And we need someone to work on azure compute, otherwise cloudmesh-ai project can't take place.

After our discussion, I have decided to replace yanting in the azure compute project since yanting prefers to work on the cloudmesh-ai service. Since azure compute project is partially finished, naturally, I can continue to work on cloudmesh-ai after completion of azure compute or do some comparison of azure compute and other cloud providers.

Yanting and Qiwei both have been notified of this issue post. They will comment if necessary.

laszewsk commented 5 years ago

great. The azure project is ongoing and a student is still working on it. see the checkins in the cloudmesh-cloud repo on azure. Missing features I am aware of are

secgroups
uploading of your own images (never been done in cloudmesh as we always use dn existing image from vendors)

The first one is very well documented and pointe out to the student.

We have a slack channel at https://app.slack.com/client/TFQC2P62D/CJWRBF1D3/thread/CFQSP8Q4T-1567738729.016200 in which that particular student reports progress. Maybe you guys can contact him his name is Joaquin

juaco77 commented 5 years ago

The Azure Provider class has implemented all the methods required by the ComputeNodeABC Abstract Class.

There are 3 methods related to key management where I got stuck:

    def keys(self):
        raise NotImplementedError

    def key_upload(self, key=None):
        raise NotImplementedError

    def key_delete(self, name=None):
        raise NotImplementedError

The main method is key_upload which takes the key from your Operating System and uploads it to the cloudmesh selected cloud.

Other clouds like openstack or aws have a pretty straightforward implementation that requires two parameters (key_name and public_key)

I.E: OPENSTACK self.cloudman.create_keypair(name, key['public_key'])

AWS self.ec2_client.import_key_pair(KeyName=key_name,PublicKeyMaterial=key['public_key'])

Unfortunately I did not find anything similar on the Azure SDK. Azure uses a template that defines each Virtual Machine set of Parameters.

I started with a structure like the one below.

vm_parameters = {
            'location': self.LOCATION,
            'os_profile': {
                'computer_name': self.VM_NAME,
                'admin_username': self.USERNAME,
                'admin_password': self.PASSWORD
            },
            'hardware_profile': {
                'vm_size': self.VM_SIZE
            },
            'storage_profile': {
                'image_reference': {
                    'publisher': publisher,
                    'offer': offer,
                    'sku': sku,
                    'version': version
                },
            },
            'network_profile': {
                'network_interfaces': [{
                    'id': nic.id,
                }]
            },
        }

When looking for a way to upload the public key I found that it is supported under the os_profile tag's linux configuration.

FOR EXAMPLE

'os_profile': {
      'computer_name': self.VM_NAME,
      'admin_username': self.USERNAME,
      "linux_configuration": {
         "disable_password_authentication": True,
         "ssh": {
             "public_keys": [{
                  "path": "/home/{}/.ssh/authorized_keys".format(USERNAME),
                  "key_data": "ssh-rsa ABCD....."
             }]
        }
     }
}

The following classes are used to assemble a full setup of vm_parameters:

The SshPublicKey class seemed to me like the equivalent implementation to AWS and Openstack.

It takes the following Parameters: path: Specifies the full path on the created VM where ssh public key is stored. If the file already exists, the specified key is appended to the file. Example: /home/user/.ssh/authorized_keys

key_data: SSH public key certificate used to authenticate with the VM through ssh. The key needs to be at least 2048-bit and in ssh-rsa format.

The difference is that it needs to be included as part of the virtual machine parameters setup.

I tried to assemble it using the following code but have not commited it since I have not been able to make it work.

from azure.mgmt.compute.v2019_03_01.models import OSProfile, LinuxConfiguration, SshConfiguration, SshPublicKey, VirtualMachine, HardwareProfile, StorageProfile, NetworkProfile, ImageReference, NetworkInterfaceReference

def create_vm_parameters(self):
        nic = self.create_nic()

        # Parse Image from yaml file
        publisher, offer, sku, version = self.default["image"].split(":")

        # Declare Virtual Machine Settings
        ssh_keys = self.key_upload(self)
        ssh_conf = SshConfiguration(public_keys=ssh_keys)
        linux_conf = LinuxConfiguration(disable_password_authentication= True, ssh=ssh_conf, provision_vm_agent=False)
        operating_system_profile = OSProfile(computer_name=self.VM_NAME, admin_username=self.USERNAME ,admin_password=self.PASSWORD,linux_configuration=linux_conf )

        hardware_prof = HardwareProfile(vm_size=self.VM_SIZE)
        image_ref    = ImageReference( publisher = publisher, offer = offer, sku = sku, version= version)
        storage_prof = StorageProfile(image_reference=image_ref)
        network_int  = NetworkInterfaceReference(id = nic.id)
        network_prof = NetworkProfile(network_interfaces=network_int)

        vm_parameters = VirtualMachine(
                            location=self.LOCATION,
                            os_profile = operating_system_profile,
                            hardware_profile = hardware_prof,
                            storage_profile = storage_prof,
                            network_profile = network_prof
                            )

        return vm_parameters

cloudmesh-community / fa19-516-157

Project selection #2