search

hashicorp / terraform

Terraform enables you to safely and predictably create, change, and improve infrastructure. It is a source-available tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned.
https://www.terraform.io
Other
42.77k stars 9.56k forks source link

azurerm - error creating vm from existing vhd - osType not supplied #6372

Closed ghost closed 8 years ago

ghost commented 8 years ago

Terraform Version

v0.6.15

Affected Resource(s)

azurerm_virtual_machine

Terraform Configuration Files

resource "azurerm_virtual_machine" "ap-service-discovery-server-0" {
    name = "ap-service-discovery-server-0"
    resource_group_name = "${var.resource_group_name}"
    location = "${var.location}"
    vm_size = "${var.instance_type}"
    network_interface_ids = ["${azurerm_network_interface.ap-service-discovery-server-0.id}"]
    availability_set_id = "${azurerm_availability_set.ap-service-discovery-availability-set.id}"

    #TODO at this point - copy from vhd storage, rename and move to VM storage acct.
    storage_os_disk {
        name = "ap-service-discovery-server-0-osdisk"
        vhd_uri = "${replace("${var.vhd_path}",".vhd","-0.vhd")}"
        create_option = "attach"
    }

    os_profile {
        computer_name = "ap-service-discovery-server-0"
        admin_username = "${var.ssh_username}"
        admin_password = "${var.ssh_password}"
        custom_data = "${template_file.user_data_0.rendered}"
    }

    os_profile_linux_config {
       disable_password_authentication = false
        /*ssh_keys {
          path = "/home/${var.ssh_username}/.ssh/authorized_keys"
          key_data = "${file("${var.ssh_key_path}")}"
        }*/
    }
}

Error

* azurerm_virtual_machine.ap-service-discovery-server-0: autorest:DoErrorUnlessStatusCode 400 PUT https://management.azure.com/subscriptions/8b086f2c-3ade-4770-8825-d97490e3fe83/resourceGroups/core-dev/providers/Microsoft.Compute/virtualMachines/ap-service-discovery-server-0?api-version=2015-06-15 failed with 400 Bad Request

from charles proxy:

"properties": {
            "statusCode": "BadRequest",
            "serviceRequestId": "2ab50479-ef29-4b24-95fe-8fd8967831cd",
            "statusMessage": "{\"error\":{\"code\":\"InvalidParameter\",\"target\":\"osDisk.osType\",\"message\":\"Required parameter 'osDisk.osType' is missing (null).\"}}"
        },

Expected Behavior

What should have happened? The VM should have successfully been created with the specified OS disk.

osType is not mentioned in the example for the virtual machine resource and no mapping seems to exist.

Actual Behaviour

error 400 bad request - see charles output above.

stack72 commented 8 years ago

Hi @tasquith

thanks for reporting the error. I am currently looking into the API for the os_type parameter. Can you tell me how you created the os_disk? This would allow me to try and create one as a test

Paul

stack72 commented 8 years ago

@tasquith

Just a FYI, the reason this wasn't added (os_type) is because it hasn't been documented on the Azure API - https://msdn.microsoft.com/en-us/library/mt163591.aspx#bk_osdisk

I will try and debug what is needed

Thanks again for pointing this out as it is a showstopper

P.

ghost commented 8 years ago

Hi @stack72,

Thanks for getting back to me so quickly!

It should just work using packer to create an image based on the openlogic 7 template.

as a side note, I have to create it using packer with the classic provider (we create a base image from openlogic, then use that to create a more specialised server - this one, I have to do this as the azurerm provider for packer doesn't yet support using a vhd as a seed to create another image), then use the cli to move it to a rm storage account, where I have to copy it x number of times using the cli , before using terraform azurerm to create x number of nodes. Is there/will there be a way of using a VHD as a source in terraform, where it then creates its own copy in a specified storage acct for each instance needed, rather than just attaching?

If you need anything more, just let me know.

Thanks,

Tom

stack72 commented 8 years ago

@tasquith

This is REALLY USEFUL info! This will help me create a good test case here - so thanks for that. Let me see if i can fix this bug and then see if we can address the second part

Paul

stack72 commented 8 years ago

Hi @tasquith

I am currently tracking this issue now as a larger set of AzureRM work. I am going to close this issue so that everything can be tracked on #6526

Paul

stack72 commented 8 years ago

Hi @tasquith

Are you in a position to be able to build terraform from source to run a test for this issue? A PR has been very helpfully created by @Erouan50 and I'd like to try and include it in 0.6.16

Paul

ghost commented 8 years ago

Hi @stack72,

I'll take a look now - progress may be slow as I've not built it before but it should be fine!

Tom

ghost commented 8 years ago

Hi @stack72,

I built the provider successfully, but I couldn't get past plan - so I assume the fix is in the core?

I tried building the whole lot just using make - but got the following issue:

ok github.com/hashicorp/terraform 0.023s ok github.com/hashicorp/terraform/builtin/bins/provider-atlas 0.019s ok github.com/hashicorp/terraform/builtin/bins/provider-aws 0.022s ok github.com/hashicorp/terraform/builtin/bins/provider-azure 0.020s ? github.com/hashicorp/terraform/builtin/bins/provider-azurerm [no test files] ? github.com/hashicorp/terraform/builtin/bins/provider-chef [no test files] ok github.com/hashicorp/terraform/builtin/bins/provider-clc 0.018s ok github.com/hashicorp/terraform/builtin/bins/provider-cloudflare 0.023s ok github.com/hashicorp/terraform/builtin/bins/provider-cloudstack 0.020s ? github.com/hashicorp/terraform/builtin/bins/provider-cobbler [no test files] ok github.com/hashicorp/terraform/builtin/bins/provider-consul 0.034s ok github.com/hashicorp/terraform/builtin/bins/provider-datadog 0.023s ok github.com/hashicorp/terraform/builtin/bins/provider-digitalocean 0.019s ok github.com/hashicorp/terraform/builtin/bins/provider-dme 0.019s ok github.com/hashicorp/terraform/builtin/bins/provider-dnsimple 0.020s ok github.com/hashicorp/terraform/builtin/bins/provider-docker 0.019s ok github.com/hashicorp/terraform/builtin/bins/provider-dyn 0.020s ok github.com/hashicorp/terraform/builtin/bins/provider-fastly 0.018s ok github.com/hashicorp/terraform/builtin/bins/provider-github 0.019s ok github.com/hashicorp/terraform/builtin/bins/provider-google 0.021s ok github.com/hashicorp/terraform/builtin/bins/provider-heroku 0.021s ok github.com/hashicorp/terraform/builtin/bins/provider-influxdb 0.020s ok github.com/hashicorp/terraform/builtin/bins/provider-librato 0.019s ok github.com/hashicorp/terraform/builtin/bins/provider-mailgun 0.021s ok github.com/hashicorp/terraform/builtin/bins/provider-mysql 0.021s ok github.com/hashicorp/terraform/builtin/bins/provider-null 0.018s ? github.com/hashicorp/terraform/builtin/bins/provider-openstack [no test files] ? github.com/hashicorp/terraform/builtin/bins/provider-packet [no test files] ok github.com/hashicorp/terraform/builtin/bins/provider-postgresql 0.020s ok github.com/hashicorp/terraform/builtin/bins/provider-powerdns 0.019s ? github.com/hashicorp/terraform/builtin/bins/provider-rundeck [no test files] ok github.com/hashicorp/terraform/builtin/bins/provider-softlayer 0.020s ? github.com/hashicorp/terraform/builtin/bins/provider-statuscake [no test files] ? github.com/hashicorp/terraform/builtin/bins/provider-template [no test files] ok github.com/hashicorp/terraform/builtin/bins/provider-terraform 0.021s ? github.com/hashicorp/terraform/builtin/bins/provider-tls [no test files] ? github.com/hashicorp/terraform/builtin/bins/provider-triton [no test files] ? github.com/hashicorp/terraform/builtin/bins/provider-ultradns [no test files] ? github.com/hashicorp/terraform/builtin/bins/provider-vcd [no test files] ok github.com/hashicorp/terraform/builtin/bins/provider-vsphere 0.028s ok github.com/hashicorp/terraform/builtin/bins/provisioner-chef 0.020s ok github.com/hashicorp/terraform/builtin/bins/provisioner-file 0.021s ok github.com/hashicorp/terraform/builtin/bins/provisioner-local-exec 0.019s ok github.com/hashicorp/terraform/builtin/bins/provisioner-remote-exec 0.019s ok github.com/hashicorp/terraform/builtin/providers/atlas 0.018s ok github.com/hashicorp/terraform/builtin/providers/aws 1.702s ok github.com/hashicorp/terraform/builtin/providers/azure 0.032s ok github.com/hashicorp/terraform/builtin/providers/azurerm 0.027s ok github.com/hashicorp/terraform/builtin/providers/chef 0.018s ok github.com/hashicorp/terraform/builtin/providers/clc 0.018s ok github.com/hashicorp/terraform/builtin/providers/cloudflare 0.018s ok github.com/hashicorp/terraform/builtin/providers/cloudstack 0.022s ok github.com/hashicorp/terraform/builtin/providers/cobbler 0.022s ok github.com/hashicorp/terraform/builtin/providers/consul 0.020s ok github.com/hashicorp/terraform/builtin/providers/datadog 0.020s ok github.com/hashicorp/terraform/builtin/providers/digitalocean 0.019s ok github.com/hashicorp/terraform/builtin/providers/dme 0.018s ok github.com/hashicorp/terraform/builtin/providers/dnsimple 0.019s ok github.com/hashicorp/terraform/builtin/providers/docker 0.019s ok github.com/hashicorp/terraform/builtin/providers/dyn 0.018s ok github.com/hashicorp/terraform/builtin/providers/fastly 0.020s ok github.com/hashicorp/terraform/builtin/providers/github 0.019s ok github.com/hashicorp/terraform/builtin/providers/google 0.034s ok github.com/hashicorp/terraform/builtin/providers/heroku 0.022s ok github.com/hashicorp/terraform/builtin/providers/influxdb 0.020s ok github.com/hashicorp/terraform/builtin/providers/librato 0.019s ok github.com/hashicorp/terraform/builtin/providers/mailgun 0.020s ok github.com/hashicorp/terraform/builtin/providers/mysql 0.018s ok github.com/hashicorp/terraform/builtin/providers/null 0.018s ok github.com/hashicorp/terraform/builtin/providers/openstack 0.022s ok github.com/hashicorp/terraform/builtin/providers/packet 0.019s ok github.com/hashicorp/terraform/builtin/providers/postgresql 0.019s ok github.com/hashicorp/terraform/builtin/providers/powerdns 0.019s ok github.com/hashicorp/terraform/builtin/providers/rundeck 0.019s ok github.com/hashicorp/terraform/builtin/providers/softlayer 0.019s ok github.com/hashicorp/terraform/builtin/providers/statuscake 0.018s ok github.com/hashicorp/terraform/builtin/providers/template 0.019s ok github.com/hashicorp/terraform/builtin/providers/terraform 0.021s ok github.com/hashicorp/terraform/builtin/providers/test 0.212s ok github.com/hashicorp/terraform/builtin/providers/tls 0.021s ok github.com/hashicorp/terraform/builtin/providers/triton 0.021s ok github.com/hashicorp/terraform/builtin/providers/ultradns 0.018s ok github.com/hashicorp/terraform/builtin/providers/vcd 0.018s ok github.com/hashicorp/terraform/builtin/providers/vsphere 0.032s ok github.com/hashicorp/terraform/builtin/provisioners/chef 0.033s ok github.com/hashicorp/terraform/builtin/provisioners/file 0.018s ok github.com/hashicorp/terraform/builtin/provisioners/local-exec 0.054s ok github.com/hashicorp/terraform/builtin/provisioners/remote-exec 0.018s ok github.com/hashicorp/terraform/command 1.446s ok github.com/hashicorp/terraform/communicator 0.019s ok github.com/hashicorp/terraform/communicator/remote 0.011s ok github.com/hashicorp/terraform/communicator/ssh 0.056s ok github.com/hashicorp/terraform/communicator/winrm 0.034s ok github.com/hashicorp/terraform/config 0.077s ok github.com/hashicorp/terraform/config/module 0.036s ok github.com/hashicorp/terraform/dag 0.014s ok github.com/hashicorp/terraform/digraph 0.011s ? github.com/hashicorp/terraform/dot [no test files] ok github.com/hashicorp/terraform/flatmap 0.012s ? github.com/hashicorp/terraform/helper/acctest [no test files] ok github.com/hashicorp/terraform/helper/config 0.018s ok github.com/hashicorp/terraform/helper/diff 0.018s ok github.com/hashicorp/terraform/helper/hashcode 0.009s ? github.com/hashicorp/terraform/helper/logging [no test files] ok github.com/hashicorp/terraform/helper/mutexkv 0.062s ok github.com/hashicorp/terraform/helper/pathorcontents 0.014s ok github.com/hashicorp/terraform/helper/resource 15.061s ok github.com/hashicorp/terraform/helper/schema 0.023s ok github.com/hashicorp/terraform/plugin 0.259s ok github.com/hashicorp/terraform/rpc 0.038s ok github.com/hashicorp/terraform/state 0.023s 2016/05/09 17:22:09 [DEBUG] PUT http://127.0.0.1:49441/api/v1/terraform/state/someuser/some-test-remote-state?access_token=sometoken&atlas_run_id= 2016/05/09 17:22:09 [DEBUG] Handling Atlas conflict response: Conflict 2016/05/09 17:22:09 [DEBUG] Atlas reported conflict, checking for equivalent states. 2016/05/09 17:22:09 [DEBUG] GET http://127.0.0.1:49441/api/v1/terraform/state/someuser/some-test-remote-state?access_token=sometoken&atlas_run_id= 2016/05/09 17:22:09 [DEBUG] States are equivalent, incrementing serial and retrying. 2016/05/09 17:22:09 [DEBUG] PUT http://127.0.0.1:49441/api/v1/terraform/state/someuser/some-test-remote-state?access_token=sometoken&atlas_run_id= 2016/05/09 17:22:09 [DEBUG] PUT http://127.0.0.1:49445/api/v1/terraform/state/someuser/some-test-remote-state?access_token=sometoken&atlas_run_id= 2016/05/09 17:22:09 [DEBUG] PUT http://127.0.0.1:49447/api/v1/terraform/state/someuser/some-test-remote-state?access_token=sometoken&atlas_run_id= 2016/05/09 17:22:09 [DEBUG] Handling Atlas conflict response: Conflict 2016/05/09 17:22:09 [DEBUG] Atlas reported conflict, checking for equivalent states. 2016/05/09 17:22:09 [DEBUG] GET http://127.0.0.1:49447/api/v1/terraform/state/someuser/some-test-remote-state?access_token=sometoken&atlas_run_id= 2016/05/09 17:22:09 [DEBUG] States are not equivalent, returning conflict. 2016/05/09 17:22:09 [DEBUG] PUT http://127.0.0.1:49450/api/v1/terraform/state/someuser/some-test-remote-state?access_token=sometoken&atlas_run_id= --- FAIL: TestAtlasClient_UnresolvableConflict (0.05s) atlas_test.go:163: Timed out after 50ms, probably because retrying infinitely. 2016/05/09 17:22:09 [ERR] PUT http://127.0.0.1:49450/api/v1/terraform/state/someuser/some-test-remote-state?access_token=sometoken&atlas_run_id= request failed: Put http://127.0.0.1:49450/api/v1/terraform/state/someuser/some-test-remote-state?access_token=sometoken&atlas_run_id=: read tcp 127.0.0.1:49451->127.0.0.1:49450: read: connection reset by peer 2016/05/09 17:22:09 [DEBUG] PUT http://127.0.0.1:49450/api/v1/terraform/state/someuser/some-test-remote-state?access_token=sometoken&atlas_run_id=: retrying in 1s 2016/05/09 17:22:10 [WARN] Ignoring AWS metadata API endpoint at default location as it doesn't return any instance-id FAIL FAIL github.com/hashicorp/terraform/state/remote 1.496s ok github.com/hashicorp/terraform/terraform 0.971s make: *** [test] Error 1

Is this a common error?

tomasquith commented 8 years ago

@tomasquith - mentioning my other account as I seem to be having issues subscribing from it.

stack72 commented 8 years ago

Hi @tasquith @tomasquith

We have gone ahead and merged this. There is no adverse effects here and normal func will continue. When 0.6.16 comes out, please can you test this works as expected and let me know how you get on :)

I am going to try and get an acceptance test in place for this

Paul

ghost commented 8 years ago

Thanks @stack72,

No problem, I'll keep an eye out for this and will test as soon as it lands.

As I can see an image_uri as part of this PR, is that a part of #6494 or am I jumping to the wrong conclusion?

Cheers,

Tom

ghost commented 8 years ago

@stack72 I've grabbed 0.6.16 from the releases service and can confirm that I can provision a machine correctly. The image uri has also allowed me to create a new VHD based on that source as expected in #6494 - I'll test that this works properly today but initial findings look good!

Thanks for all of the help on these!

Tom

stack72 commented 8 years ago

@tasquith

This is fantastic news. I am very pleased that this has unblocked you

Paul

imduffy15 commented 8 years ago

@tasquith How did you use terraform with charles proxy and get around invalid ssl errors?

ghost commented 8 years ago

@imduffy15

Hi Ian,

All I do is start charles proxy, perform the following steps to trust the certs:

In Charles go to the Help menu and choose "SSL Proxying > Install Charles Root Certificate". Keychain Access will open, and prompt you about the certificate. Click the "Always Trust" button. You will then be prompted for your Administrator password to update the system trust settings.

https://www.charlesproxy.com/documentation/using-charles/ssl-certificates/

Then:

You must specifically identify the host names you want to enable SSL Proxying on. The list is in the Proxy Settings, SSL tab. You can also right-click on a host name in the structure view and turn on or off SSL Proxying.

After adding a host name to the SSL Proxying list you may need to restart Charles for existing browser sessions to change.

If you want to SSL Proxy all host names then enter * into the host names list in the SSL Proxying Settings.

https://www.charlesproxy.com/documentation/proxying/ssl-proxying/

Once that's done, you should start to see the api calls to azure, which give you more detailed errors.

Cheers, Tom

imduffy15 commented 8 years ago

Thank you so much!

On Friday 17 June 2016, Tom Asquith notifications@github.com wrote:

@imduffy15 https://github.com/imduffy15

Hi Ian,

All I do is start charles proxy, perform the following steps to trust the certs:

In Charles go to the Help menu and choose "SSL Proxying > Install Charles Root Certificate". Keychain Access will open, and prompt you about the certificate. Click the "Always Trust" button. You will then be prompted for your Administrator password to update the system trust settings.

https://www.charlesproxy.com/documentation/using-charles/ssl-certificates/

Then:

You must specifically identify the host names you want to enable SSL Proxying on. The list is in the Proxy Settings, SSL tab. You can also right-click on a host name in the structure view and turn on or off SSL Proxying.

After adding a host name to the SSL Proxying list you may need to restart Charles for existing browser sessions to change.

If you want to SSL Proxy all host names then enter * into the host names list in the SSL Proxying Settings.

https://www.charlesproxy.com/documentation/proxying/ssl-proxying/

Once that's done, you should start to see the api calls to azure, which give you more detailed errors.

Cheers, Tom

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/hashicorp/terraform/issues/6372#issuecomment-226733985, or mute the thread https://github.com/notifications/unsubscribe/ABL5K3ZIRzliJT_yauvqbO5tfb4vY84bks5qMnRwgaJpZM4IQ2Rn .

stack72 commented 8 years ago

@imduffy15 / @tasquith

Terraform 0.7 will expose much better error messages from the SDK so don't spend too much time on this ;) it's a temp thing

P.

imduffy15 commented 8 years ago

OK for anyone else... on the latest version of OSX this isn't as straight forward as @tasquith described on the latest version of OSX.

The SSL library within golang uses /usr/bin/security find-certificate -a -p /System/Library/Keychains/SystemRootCertificates.keychain to discover certificates available to use. On The latest version of OSX you cannot add certificates to the system root keychain.

You must boot your mac into recovery mode, open the terminal from the utilities menu, execute csrutil disable which disables some core osx security stuff.

Then you can reboot and execute sudo security add-trusted-cert -k /System/Library/Keychains/SystemRootCertificates.keychain charles-ssl-proxying-certificate.crt which will add the certificate.

ghost commented 8 years ago

@imduffy15

Cheers for the info! I'm intrigued, I'm using El Capitan which I thought was the latest and didn't have to do any of that!

Either way, all that matters is that you've got it working!

mcheriyath commented 8 years ago

Can someone share the actual code used terraform which worked for using an existing OS disk to create a VM without OS_profile ?

bbaassssiiee commented 7 years ago

You don't need to add to the system root keychain to trust root certificates that are not in there yet. You can ad them to your own (login.keychain) with a flag -r trustRoot

security add-trusted-cert -r trustRoot -p ssl -k "{{ HOME }}/Library/Keychains/login.keychain-db" "myOwnRootCA.cer"

github-actions[bot] commented 3 years ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.