search

danielsollondon / azvmimagebuilder

Azure VM Image Builder
180 stars 153 forks source link

ALERT - Service Build Errors in "West Central US", "West US", "West US 2", "South Central US #101

Closed danielsollondon closed 3 years ago

danielsollondon commented 3 years ago

We are experiencing some intermittent issues, symptoms, you can start the build, but the build fails with a "During the image build a failure has occurred. Please review the build log to identify which build/customization step failed. For more troubleshooting steps go to https://aka.ms/azvmimagebuilderts. Image...". When you check the storage account for the customization.log, you do not see any logs. Apologies, we are working on this now, the current mitigation is to use an alternative region, such as eastus.

Next update 1030am Pacific

danielsollondon commented 3 years ago

Hi All - The dev team are still working on this as high priority, no current updates on resolution, but I will update again at 3pm Pacific. Thank you for your patience, and apologies for the inconvenience.

danielsollondon commented 3 years ago

Hi All, we have mitigated the issue, you can use the regions again.

We will be doing an RCA over the coming days, and will be implementing learnings. Again, apologies for the inconvenience.

Please let us know if you are experiencing further issues.

Kind regards,

BladeFireLight commented 3 years ago

I am again seeing random issues with westcentralus but no problems in eastus. issues like AIB getting stuck looping this in the log PACKER OUT azure-arm: IMAGE_STATE_COMPLETE

or never creating the IT_* RG and being left in a running state.

danielsollondon commented 3 years ago

@BladeFireLight - are you still seeing issues? If so can you raise a case here and I will triage this with the CSS team please? https://github.com/danielsollondon/azvmimagebuilder/blob/master/troubleshootingaib.md#troubleshooting-azure-vm-image-build-aib-failures