search

hashicorp / packer-plugin-openstack

Packer plugin for OpenStack Builder
https://www.packer.io/docs/builders/openstack
Mozilla Public License 2.0
21 stars 19 forks source link

Openstack image gets deleted after their creation with packer #37

Open hc-github-team-packer opened 3 years ago

hc-github-team-packer commented 3 years ago

This issue was originally opened by @unnikrishnan262 in https://github.com/hashicorp/packer/issues/11136 and has been migrated to this repository. The original issue description is below.

When filing a bug, please include the following headings if possible. Any example text in this template can be deleted.

Overview of the Issue

I encounter the same issue as the one described here (7312). (which is closed - i've tested all the workaround shown there)

Every time I create an image on Openstack, it is get deleted at the end of the process. Here is the basic output of the packer command. We are using VMware as OpenStack backend. If I run the packer build with '-debug' option, the image creation is successful.

It looks like from the Nova logs that there is a race condition in which vmware snapshot get deleted before the image upload completes and when we run with debug, there are few secs delay while we hit enter multiple time. Can we introduce a pause before the "Terminating the source server" step or do more check to make sure image is created successfully?

it reads like packer only looks at the image and when it's available, while there's an endpoint in Nova for checking if an asynchronous task is done. https://docs.openstack.org/api-guide/compute/faults.html#server-actions 

openstack: [DEBUG] Request GET https://image-3.ap-ae-1.cloud.sap/v2/images/5404d85a-aef5-42d2-ab5b-5562f86e11e6 200
openstack: [DEBUG] Request GET https://image-3.ap-ae-1.cloud.sap/v2/images/5404d85a-aef5-42d2-ab5b-5562f86e11e6 200

==> openstack: Pausing after run of step 'stepCreateImage'. Press enter to continue. ==> openstack: Pausing after run of step 'stepUpdateImageTags'. Press enter to continue. ==> openstack: Pausing after run of step 'stepUpdateImageVisibility'. Press enter to continue. ==> openstack: Pausing after run of step 'stepAddImageMembers'. Press enter to continue. ==> openstack: Pausing after run of step 'stepUpdateImageMinDisk'. Press enter to continue. ==> openstack: Pausing before cleanup of step 'stepUpdateImageMinDisk'. Press enter to continue. ==> openstack: Pausing before cleanup of step 'stepAddImageMembers'. Press enter to continue. ==> openstack: Pausing before cleanup of step 'stepUpdateImageVisibility'. Press enter to continue. ==> openstack: Pausing before cleanup of step 'stepUpdateImageTags'. Press enter to continue. ==> openstack: Pausing before cleanup of step 'stepCreateImage'. Press enter to continue. ==> openstack: Pausing before cleanup of step 'StepDetachVolume'. Press enter to continue. ==> openstack: Pausing before cleanup of step 'StepStopServer'. Press enter to continue. ==> openstack: Pausing before cleanup of step 'StepCleanupTempKeys'. Press enter to continue. ==> openstack: Pausing before cleanup of step 'StepProvision'. Press enter to continue. ==> openstack: Pausing before cleanup of step 'StepConnect'. Press enter to continue. ==> openstack: Pausing before cleanup of step 'StepAllocateIp'. Press enter to continue. ==> openstack: Pausing before cleanup of step 'StepWaitForRackConnect'. Press enter to continue. ==> openstack: Pausing before cleanup of step 'StepGetPassword'. Press enter to continue. ==> openstack: Pausing before cleanup of step 'StepRunSourceServer'. Press enter to continue. ==> openstack: Terminating the source server: 33e859df-d6c6-4885-b383-4c7fb3a1dd99 ... openstack: [DEBUG] Request POST https://compute-3.ap-ae-1.cloud.sap:443/v2.1/servers/33e859df-d6c6-4885-b383-4c7fb3a1dd99/action 202 openstack: [DEBUG] Request GET https://compute-3.ap-ae-1.cloud.sap:443/v2.1/servers/33e859df-d6c6-4885-b383-4c7fb3a1dd99 200 openstack: [DEBUG] Request GET https://compute-3.ap-ae-1.cloud.sap:443/v2.1/servers/33e859df-d6c6-4885-b383-4c7fb3a1dd99 200

[root@test100 ~]# /usr/local/bin/packer2 build -on-error=ask -force rhel.json openstack: output will be in this color.

==> openstack: Loading flavor: 20 openstack: Verified flavor. ID: 20 ==> openstack: Creating temporary keypair: packer_60e3fa23-14b7-398b-3c79-61a5899874f6 ... ==> openstack: Created temporary keypair: packer_60e3fa23-14b7-398b-3c79-61a5899874f6 ==> openstack: Launching server... ==> openstack: Launching server... openstack: Server ID: dfd3de80-6f6e-4398-8fbd-c0e83f802b49 ==> openstack: Waiting for server to become ready... openstack: Floating IP not required ==> openstack: Using SSH communicator to connect: 10.209.251.152 ==> openstack: Waiting for SSH to become available... ==> openstack: Connected to SSH! ==> openstack: Stopping server: dfd3de80-6f6e-4398-8fbd-c0e83f802b49 ... openstack: Waiting for server to stop: dfd3de80-6f6e-4398-8fbd-c0e83f802b49 ... ==> openstack: Creating the image: rhel-6-vmware openstack: Image: f01cbcb0-ea30-4cac-a3bb-f7c2ca747c70 ==> openstack: Waiting for image rhel-6-vmware (image id: f01cbcb0-ea30-4cac-a3bb-f7c2ca747c70) to become ready... ==> openstack: Terminating the source server: dfd3de80-6f6e-4398-8fbd-c0e83f802b49 ... ==> openstack: Deleting temporary keypair: packer_60e3fa23-14b7-398b-3c79-61a5899874f6 ... Build 'openstack' finished after 5 minutes 12 seconds.

==> Wait completed after 5 minutes 12 seconds

==> Builds finished. The artifacts of successful builds are: --> openstack: An image was created: f01cbcb0-ea30-4cac-a3bb-f7c2ca747c70 [root@test100 ~]#

Packer version

1.6.6

Simplified Packer Buildfile

If the file is longer than a few dozen lines, please include the URL to the gist of the log or use the Github detailed format instead of posting it directly in the issue.

Operating system and Environment details

I tried running packer on my Mac, RHEL7 but same result.

The Openstack nova log shows the below error.

2021-07-01 06:59:32,511.511 6 ERROR oslo_messaging.rpc.server [req-9e3cd3c3-b2cc-471b-9d1d-940ec32ec1e3 greq-522d7578-a4db-f848-f209-788fb4149167 fdda6904f2d786d6092ca89c0e991d96f9e823fac746ed4ea635ab0199cea8ae 184f96ee29874bb488574a0fc130b094 - 4f18f794e1f04441a4607200295b7f2e 4f18f794e1f04441a4607200295b7f2e] Exception during message handling: ManagedObjectNotFoundException: The object 'vim.vm.Snapshot:snapshot-2028228' has already been deleted or has not been completely created

nywilken commented 3 years ago

Hi @unnikrishnan262 thanks for opening up this issue with the detailed debug information. I've moved this issue to the OpenStack plugin repo so that it can be tracked here.

With that said, I see that you are running version 1.6.6 of Packer. There have been some changes to the OpenStack builder since versions 1.7.0 and 1.7.1 of Packer.

Does running with the latest version of Packer result in the same error?

unnikrishnan262 commented 3 years ago

Thank you for the update. I will test the same with latest code and update

unnikrishnan262 commented 3 years ago

Tested the same code with 1.7.3 and running into the same issue.

michaelw730 commented 1 month ago

@unnikrishnan262 . I know this is old but did you get a solution for this?

I'm experiencing the same issue but it looks like it's to do with the post-processor. When the post-processor is used, the image is deleted. The docs suggest keep_input_artifact = true to keep the image but it doesn't work.