"vagrant destroy" does not destroy SoftLayer virtual server

[irifed]

Hi, I've noticed today that vagrant destroy command stopped destroying virtual servers on SoftLayer:

[irina@neuron cluster-ampcamp]$  vagrant up --provider=softlayer --no-provision
Bringing machine 'master' up with 'softlayer' provider...
==> master: Creating a new SoftLayer instance...
==> master: Waiting for instance provisioning. This may take a few minutes...
==> master: SoftLayer instance successfully provisioned!
==> master: Waiting for machine to boot. This may take a few minutes...
    master: SSH address: 50.22.13.215:22
    master: SSH username: root
    master: SSH auth method: private key
==> master: Machine booted and ready!
==> master: Rsyncing folder: /Users/irina/Projects/cloud/vagrant_machines/cluster-ampcamp/ => /vagrant
==> master: Machine not provisioning because `--no-provision` is specified.
vagrant up --provider=softlayer --no-provision  20.27s user 2.70s system 2% cpu 13:18.85 total
[irina@neuron cluster-ampcamp]$ vagrant status                                
Current machine states:

master                    running (softlayer)

The SoftLayer instance is running. To stop this machine, you can run
`vagrant halt`. To destroy the machine, you can run `vagrant destroy`.
[irina@neuron cluster-ampcamp]$ sl vs list | grep "50.22.13.215"             
7515982  dal05  master.ampcamp.irina.com                              1   1024   50.22.13.215     10.84.146.139   NULL  i.fedulova
[irina@neuron cluster-ampcamp]$ vagrant destroy -f
==> master: Destroying the SoftLayer instance...
==> master: Running cleanup tasks for 'ansible' provisioner...
[irina@neuron cluster-ampcamp]$ vagrant status                                                                                                                                      
Current machine states:

master                    not created (virtualbox)

The environment has not yet been created. Run `vagrant up` to
create the environment. If a machine is not created, only the
default provider will be shown. So if a provider is not listed,
then the machine is not created for that environment.
[irina@neuron cluster-ampcamp]$ sl vs list | grep "50.22.13.215"                                                                                                                    
7515982  dal05  master.ampcamp.irina.com                              1   1024   50.22.13.215     10.84.146.139   NULL  NULL

After vagrant destroy vagrant status reports that machine is not created, no errors were displayed but sl vs list command shows that virtual server is running. Previously I did not have problems with vagrant destroy. Am I missing something?..

Vagrant 1.6.5
vagrant-softlayer (0.4.0)

[ju2wheels]

Hi @irifed, can you verify there is no pending transaction for that server in the portal. I dont think the destroy actually waits for the transaction if I recall correctly, ill have to double check later, so it could be a delayed destroy on SL side.

[underscorephil]

@irifed @ju2wheels: You are correct. The cancellation request will fail if there is an active transaction already on the target VM.

The API should return an error to let you know there is an active transaction.

SoftLayer.exceptions.SoftLayerAPIError: SoftLayerAPIError(SoftLayer_Exception_Public): This cancellation could not be processed please contact support.This cancellation could not be processed please contact support. There is currently an active transaction.

If this is being caused by an active transaction you will have to have the client code wait and retry, or have the user retry at a later time.

[ju2wheels]

I deleted a vm this morning myself and just went back to the portal to verify that it was gone and its not, however I cant view the details of that vm in the portal either, I get:

Computing instance (vagranttest5.<mydomain>) cannot be upgraded due to the incorrect billing information. Please contact sales.

I wasnt actually trying to upgrade it over the portal, just clicking the vm hostname link to show the details from the Computing listing page.

@underscorephil: any exceptions from the API would have been raised and reported by vagrant in the above case. I also didnt get any exception when deleting my vm so I dont think its related to any pending transaction on the vm.

[ju2wheels]

I opened Ticket # 15610512, I get the above error in the manage.softlayer.com portal when trying to view the deleted vm details and in the control.softlayer.com portal it does show the vm details but it does not allow me to delete it even though I have pretty much all permissions available on my account.

The API itself is not returning any exceptions.

[underscorephil]

@ju2wheels @irifed I took a peek at the ticket and worked with our support and customer service departments a bit on this. It seems that what you have run into is an edge case where in some situations a cancellation will take longer than expected to process and the virtual guest object is not removed until the process is complete. Our support group has escalated this to the dev team and hopefully we may be able to find an elegant way to handle these situations.

The specific server mentioned in the ticket should be canceled fully now.

[irifed]

@underscorephil , thanks - I confirm that vagrant destroy is working normally today