Azure / AppConfiguration

Questions, feedback and samples for Azure App Configuration service
MIT License
241 stars 73 forks source link

A task is missing. The pipeline references a task called 'AzureAppConfigurationPush' with version '6'. This indicates that the specific task version can not be found #959

Closed concreteutopia closed 2 months ago

concreteutopia commented 2 months ago

Edit: Resolution guidance can be found in this comment

Receiving the task is missing error with versions 5, 6 and 7 of 'AzureAppConfigurationPush starting around 3pm EST. Occurring in multiple Azure DevOps projects / orgs and pipelines. No changes were made to our pipelines around this time.

A task is missing. The pipeline references a task called 'AzureAppConfigurationPush' with version '6'. This indicates that the specific task version can not be found

ez5504 commented 2 months ago

This just started happening for me as well about an hour ago. I only tried changing from v6 to v7 but saw the same result. Doesn't seem like the task has been updated, nor any changes to our ADO instance. It's blocking all our builds from running right now.

lettucemode commented 2 months ago

Same issue for us with the AzureAppConfiguration task. Was there a recent deployment or change?

image

adrian-patt commented 2 months ago

Same here. Our pipeline using this task is blocked because of this.

A task is missing. The pipeline references a task called 'AzureAppConfiguration' with version '6'. This indicates that the specific task version can not be found.

EDIT: Was this task somehow removed from the built-in tasks? Many stack overflow posts reference disabled built-in tasks as the source of this issue:

MaryanneNjeri commented 2 months ago

Hello currently looking into this

lettucemode commented 2 months ago

I have worked around the issue by changing the version to 10 instead of 6. Build is proceeding per usual.

I noticed that the only available version of these tasks in the official repo is v10: https://github.com/microsoft/azure-pipelines-tasks/tree/master/Tasks

And it looks like these tasks were officially adopted into the official repo a few weeks ago: https://github.com/microsoft/azure-pipelines-tasks/pull/20328 https://github.com/microsoft/azure-pipelines-tasks/pull/20338

But we also have the ADO extension installed from the marketplace (for some reason, this was set up before I got here): https://marketplace.visualstudio.com/items?itemName=AzureAppConfiguration.azure-app-configuration-task&targetId=ca8b8e9e-7645-44ce-bbb0-091e4afb6830&utm_source=vstsproduct&utm_medium=ExtHubManageList

To hazard a guess - there was an update/deployment to include these tasks in the built-in set, and doing so overrides the extension such that the extension's task versions are no longer usable?

bbrandt commented 2 months ago

Release notes for v7 and lower are here: https://github.com/Azure/AppConfiguration/blob/main/releaseNotes/AzureDevOpsPipelineExtension.md

But this page lists a v8, so maybe App Config team broke something trying to switch it from a built-in task to a task on the market? https://marketplace.visualstudio.com/items?itemName=AzureAppConfiguration.azure-app-configuration-task

MaryanneNjeri commented 2 months ago

Update Sept 5, 11:43 EST

Version 10 of the AppConfiguration and AppConfigurationPush tasks has been removed. This should restore the functionality of previous versions of these respective tasks.

Those who used the workaround mentioned below should revert back to the version of the task they were using previously.

Those who changed their name to the long form of the task, e.g. "AzureAppConfiguration.azure-app-configuration-task-push.custom-build-release-task.AzureAppConfigurationPush", should revert back to the short form, e.g. "AzureAppConfigurationPush"

We apologize for the inconvenience given our previously recommended workaround to upgrade to version 10.

Latest Versions

AzureAppConfiguration: 8 AzureAppConfigurationPush: 7

Issue

We recently released version 10 of the AppConfiguration and AppConfigurationPush tasks and this is causing older versions of each of these tasks to appear missing in some cases.

~## Workaround~ ~To use the old task specify the task name as~ ~ AzureAppConfiguration.azure-app-configuration-task-push.custom-build-release-task.AzureAppConfigurationPush for the AppConfigurationPush task.~ ~ AzureAppConfiguration.azure-app-configuration-task-push.custom-build-release-task.AzureAppConfiguration for the AppConfiguration task.~

~Upgrade to version 10 of the following tasks~ ~ AppConfigurationPush~ ~ AppConfiguration~

We are currently looking into resolving the issue for older versions. We are also working on updating our release notes to reflect the latest updates.

bbrandt commented 2 months ago

Yes. Thanks. Updating to AzureAppConfiguration@10 as noted by @MaryanneNjeri and @lettucemode worked for me as well.

cheesi commented 2 months ago

The classic editor give some enlightenment. The task is there two times: image

Once containing version 3-7 And once containing version 10

dawidholka commented 2 months ago

I was able to use old task by specifying this task name: AzureAppConfiguration.azure-app-configuration-task-push.custom-build-release-task.AzureAppConfigurationPush@3

MaryanneNjeri commented 2 months ago

After doing some investigation we also realized that the extension task name somehow got updated to AzureAppConfiguration.azure-app-configuration-task.custom-build-release-task.AzureAppConfiguration after the version10 task was deployed which I suspect could be causing the extension task to go missing.

We are currently looking into resolving the issue for the missing task for older versions

rwonepipeline commented 2 months ago

AzureAppConfiguration.azure-app-configuration-task-push.custom-build-release-task.AzureAppConfigurationPush@3

Hello, i am not able to use the old task specifying the task name as you mentioned. i get the same error task is missing I am using the task in yaml defined pipelines Updating to AppConfigurationPush to v10 this error Key cannot start with the reserved prefix for feature flags. - I believe the support for prefix .appconfig.featureflag/xxx is removed from version 5. since we don't have time now to remove the prefix , it would be nice if someone can help with a workaround to use versions less than 5

dawidholka commented 2 months ago

AzureAppConfiguration.azure-app-configuration-task-push.custom-build-release-task.AzureAppConfigurationPush@3

Hello, i am not able to use the old task specifying the task name as you mentioned. i get the same error task is missing I am using the task in yaml defined pipelines Updating to AppConfigurationPush to v10 this error Key cannot start with the reserved prefix for feature flags. - I believe the support for prefix .appconfig.featureflag/xxx is removed from version 5. since we don't have time now to remove the prefix , it would be nice if someone can help with a workaround to use versions less than 5

Make sure you have this extension installed: https://marketplace.visualstudio.com/items?itemName=AzureAppConfiguration.azure-app-configuration-task-push

rwonepipeline commented 2 months ago

@dawidholka

Today we have already uninstalled and installed it once again our Azure DevOps server. seems like the older version might have gone and only the recent version is available now in our server

the latest version for Azuer App Configuration Push task is V7.0.0 . but in my pipeline i use v10 and it works , though with the error, how does it works?

Tiberriver256 commented 2 months ago

*NOTE: This breaks redeploy functionality. So you will no longer be able to re-run old pipelines :(

concreteutopia commented 2 months ago

The @10 workaround is no longer working. Getting a few different errors:

[warning]Failed to download task 'AzureAppConfigurationPush'. Error No task definition found matching ID 04fc3ed1-4513-4f38-b0fe-5a15864bfa9c and version 10.245.0. You must register the task definition before uploading the package.

[error]wrong number of segments

Tiberriver256 commented 2 months ago

Same... we were in the middle of updating all of our pipelines to 10. Has the update been pulled? @MaryanneNjeri

MattJeanes commented 2 months ago

Yep, this is what we're seeing as of a few minutes ago: ##[error]A task is missing. The pipeline references a task called '04fc3ed1-4513-4f38-b0fe-5a15864bfa9c'. This usually indicates the task isn't installed, and you may be able to install it from the Marketplace: https://marketplace.visualstudio.com. (Task version 10.245.0, job 'Job', step 'AzureAppConfigurationPush'.)

jimmyca15 commented 2 months ago

Version 10 has been removed. This should cause previous versions of the AzureAppConfiguration and AzureAppConfigurationPush tasks to execute successfully.

Those who upgraded to version 10 will have to revert back to the version they were using previously.

We apologize for the inconvenience given our previously recommended workaround to upgrade to version 10.

evertonlsouza commented 2 months ago

Yesterday my pipelines were running with version 6 and started giving errors, I changed it to version 10 and it started working, now my pipes are not working with any version, I tested 5, 6, 7 and 8 and it is returning the following

error: ## [error]wrong number of segments.

Anyone else having this problem?

jimmyca15 commented 2 months ago

@evertonlsouza

How are you referencing the AzureAppConfiguration task?

For example

- task: AzureAppConfiguration@7
danieljaime-liberis commented 2 months ago

We're using - task: AzureAppConfigurationPush@x where x used to be 5, now it is 10 and broken

jimmyca15 commented 2 months ago

@danieljaime-liberis

Version 10 has been removed, if you revert back to 5 does it work again?

evertonlsouza commented 2 months ago

Hi, I made a change that was sent by MaryanneNjeri, and it worked, it seems that the name of the task also changed, making it work like this:

danieljaime-liberis commented 2 months ago

@jimmyca15 I just tried kicking it off as you were responding. The pipeline didn't immediately fail so we'll see if the actual task runs and passes in a few mins. It is currently - task: AzureAppConfigurationPush@5

Edit: The task ran find now I rolled back to 5.

jimmyca15 commented 2 months ago

@evertonlsouza

Please revert back to the short name

task: AzureAppConfiguration@6

Tiberriver256 commented 2 months ago

Rolling forward - AzureAppConfigurationPush@10 No longer works Rolling back - AzureAppConfigurationPush@3 (what we were on) no longer works

AzureAppConfigurationPush@7 seems to be the only thing that works

MaryanneNjeri commented 2 months ago

@Tiberriver256 what error are you getting after rolling back to AzureAppConfigurationPush@3 ? Which Authentication method is the service connection using ?

evertonlsouza commented 2 months ago

@jimmyca15, It started working the way it was before.

AzureAppConfiguration@6

jimmyca15 commented 2 months ago

@Tiberriver256

We have updated the guidance here to reflect this. Version 10 had to be removed to restore the functionality of previous versions.

jimmyca15 commented 2 months ago

@evertonlsouza

Thank you for confirming.

MattJeanes commented 2 months ago

Can you confirm what the latest version is of both the AzureAppConfiguration and AzureAppConfigurationPush tasks that we should use now please?

jimmyca15 commented 2 months ago

AzureAppConfiguration: 8 AzureAppConfigurationPush: 7

@MattJeanes

rwonepipeline commented 2 months ago

@MaryanneNjeri After updating to AzureAppConfigurationPush: 7, we get Key cannot start with the reserved prefix for feature flags error

jimmyca15 commented 2 months ago

@rwonepipeline

What version were you using previously?

rwonepipeline commented 2 months ago

@jimmyca15 we were using AzureAppConfigurationPush: 3

Tiberriver256 commented 2 months ago

@MaryanneNjeri We are getting this error with AzureAppConfigurationPush@3:

image

MaryanneNjeri commented 2 months ago

After updating to AzureAppConfigurationPush: 7, we get Key cannot start with the reserved prefix for feature flags error

@rwonepipeline a breaking change was introduced in version 5, please see here where Settings with feature flag prefix and content type will fail to import.

MaryanneNjeri commented 2 months ago

@Tiberriver256 are you referencing the task as AzureAppConfigurationPush@3 ? From the errors it also seems that version 10 of the push task is still being refernced

rwonepipeline commented 2 months ago

@MaryanneNjeri Thanks, Seems like we have to make too many changes at least from the content part. is it possible to use older version say zureAppConfigurationPush@3. from marketplace, we can only download the latest version of that particular task

jimmyca15 commented 2 months ago

@Tiberriver256 @rwonepipeline

Did you happen to uninstall and re-install the AzureAppConfiguration market place extension?

rwonepipeline commented 2 months ago

@Tiberriver256 @rwonepipeline

Did you happen to uninstall and re-install the AzureAppConfiguration market place extension?

unfortunately yes, When I saw V10 to be used :D

Tiberriver256 commented 2 months ago

@Tiberriver256 @rwonepipeline

Did you happen to uninstall and re-install the AzureAppConfiguration market place extension?

We did unfortunately as a troubleshooting step early on 🥲

Werner-Swart-83 commented 2 months ago

We just spent the day moving to 10 to find it removed? This is utterly disruptive and causing unnecessary costs.

holyyyns commented 2 months ago

In the first place you released breaking change without any notification, then we spent hours changing all pipelines to @10 and now we come back 4 hours later and you remove @10 and we have to revert back to either @6 or @8... How the hell did you manage to pull that off?

davenicoll commented 2 months ago

Couldn't the last working version have been promoted into v10 and all of this avoided?

jimmyca15 commented 2 months ago

On behalf of the App Configuration team, I apologize for the disruption that this issue has caused. I understand that we provided a work around, and then the work around became non-operable resulting in additional disruption and lost time. We do not take this issue lightly. We strive to be a resilient, reliable configuration solution for developers and that includes being resilient and reliable in dev ops scenarios. We cannot make up for the lost hours caused by this particular issue, but we can ensure that every bit of attention possible is being put in place to ensure that this is never repeated. This wasn't the result of a planned breaking change. Details on the root cause will follow.

@cheesi @davenicoll @holyyyns @Werner-Swart-83 @rwonepipeline @Tiberriver256 @MattJeanes @concreteutopia @bbrandt @lettucemode @adrian-patt @ez5504 @dawidholka

&Everyone else impacted

MaryanneNjeri commented 2 months ago

Root Cause

Prior to September 4th, the AzureAppConfiguration (AAC) and AzureAppConfigurationPush (AACP) tasks were available as Azure Dev Ops market place extensions. Their latest versions were v8 and v7 respectively. On September 4th 7:38 PM UTC, The App Configuration team released new versions of these tasks, v10, as built-in tasks in Azure Dev Ops, rather than market place extensions. Upon release, it was discovered that v10 was working, yet all previous versions of the task were no longer operable. A testing deficiency for the migration to built-in tasks caused this issue to go undetected before release. At this point, an incorrect initial determination was made that removing the newly released v10 was not an option to restore the functionality of previous versions. The App Configuration team immediately began working to restore the operability of previous task versions, e.g. AAC v8 and below and AACP v7 and below. As the team worked to restore previous versions of the task, it was recommended to migrate to v10 to enable tasks to run successfully. During this effort it was determined that restoring the functionality of previous task versions would take longer than originally estimated, and additionally it was verified that removing v10 would restore previous versions contrary to previous conclusions. To avoid further impact on those who were unable to execute the work around, the team removed v10 of the AAC and AACP tasks. The removal of v10 completed on September 5th 2:40 PM UTC. This resulted in any pipelines that were updated to use the workaround to become inoperable. In order to avoid further disruption, v10 was not re-released in any capacity.

holyyyns commented 2 months ago

Root Cause

Prior to September 4th, the AzureAppConfiguration (AAC) and AzureAppConfigurationPush (AACP) tasks were available as Azure Dev Ops market place extensions. Their latest versions were v8 and v7 respectively. On September 4th 7:38 PM UTC, The App Configuration team released new versions of these tasks, v10, as built-in tasks in Azure Dev Ops, rather than market place extensions. Upon release, it was discovered that v10 was working, yet all previous versions of the task were no longer operable. A testing deficiency for the migration to built-in tasks caused this issue to go undetected before release. At this point, an incorrect initial determination was made that removing the newly released v10 was not an option to restore the functionality of previous versions. The App Configuration team immediately began working to restore the operability of previous task versions, e.g. AAC v8 and below and AACP v7 and below. As the team worked to restore previous versions of the task, it was recommended to migrate to v10 to enable tasks to run successfully. During this effort it was determined that restoring the functionality of previous task versions would take longer than originally estimated, and additionally it was verified that removing v10 would restore previous versions contrary to previous conclusions. To avoid further impact on those who were unable to execute the work around, the team removed v10 of the AAC and AACP tasks. The removal of v10 completed on September 5th 2:40 PM UTC. This resulted in any pipelines that were updated to use the workaround to become inoperable. In order to avoid further disruption, v10 was not re-released in any capacity.

Hello Crowdstrike...

MaryanneNjeri commented 2 months ago

I would also like to extend my sincere apologies to everyone affected by the recent disruption. We understand the inconvenience this has caused and deeply regret any impact it may have had on your operations.