search

Azure / apiops

APIOps applies the concepts of GitOps and DevOps to API deployment. By using practices from these two methodologies, APIOps can enable everyone involved in the lifecycle of API design, development, and deployment with self-service and automated tools to ensure the quality of the specifications and APIs that they’re building.
https://azure.github.io/apiops
MIT License
321 stars 187 forks source link

[BUG] Error in Publisher when making a new revision current #184

Closed anotherRedbeard closed 3 months ago

anotherRedbeard commented 1 year ago

Release version

v.3.0.1

Describe the bug

When I create a new revision in my API and extract and publish the changes, everything works as expected, but when I make that revision current, extract and publish the changes I get the following error attempting to publish from the last commit.

Expected behavior

Revisions should match lower level environments and have the same version id as the current revision.

Actual behavior

I'm not 100% sure what's happening, but it appears that when you make the revision current it changes the url which is changing the folder in the extractor run.

Here is the error:

crit: Publisher[0]
      System.ArgumentException: An element with the same key but a different value already exists. Key: 'Put'
         at System.Collections.Immutable.ImmutableDictionary`2.HashBucket.Add(TKey key, TValue value, IEqualityComparer`1 keyOnlyComparer, IEqualityComparer`1 valueComparer, KeyCollisionBehavior behavior, OperationResult& result)
         at System.Collections.Immutable.ImmutableDictionary`2.AddRange(IEnumerable`1 items, MutationInput origin, KeyCollisionBehavior collisionBehavior)
         at System.Collections.Immutable.ImmutableDictionary`2.AddRange(IEnumerable`1 pairs, Boolean avoidToHashMap)
         at System.Collections.Immutable.ImmutableDictionary`2.AddRange(IEnumerable`1 pairs)
         at System.Collections.Immutable.ImmutableDictionary.ToImmutableDictionary[TSource,TKey,TValue](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 keyComparer, IEqualityComparer`1 valueComparer)
         at System.Collections.Immutable.ImmutableDictionary.ToImmutableDictionary[TSource,TKey,TValue](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector)
         at publisher.Publisher.GetCommitIdFiles(CommitId commitId)
         at publisher.Publisher.RunWithCommitId(CommitId commitId, CancellationToken cancellationToken)
         at publisher.Publisher.Run(CancellationToken cancellationToken)
         at publisher.Publisher.ExecuteAsync(CancellationToken cancellationToken)
info: Microsoft.Hosting.Lifetime[0]
      Application is shutting down...
fail: Microsoft.Extensions.Hosting.Internal.Host[9]
      BackgroundService failed
      System.ArgumentException: An element with the same key but a different value already exists. Key: 'Put'
         at System.Collections.Immutable.ImmutableDictionary`2.HashBucket.Add(TKey key, TValue value, IEqualityComparer`1 keyOnlyComparer, IEqualityComparer`1 valueComparer, KeyCollisionBehavior behavior, OperationResult& result)
         at System.Collections.Immutable.ImmutableDictionary`2.AddRange(IEnumerable`1 items, MutationInput origin, KeyCollisionBehavior collisionBehavior)
         at System.Collections.Immutable.ImmutableDictionary`2.AddRange(IEnumerable`1 pairs, Boolean avoidToHashMap)
         at System.Collections.Immutable.ImmutableDictionary`2.AddRange(IEnumerable`1 pairs)
         at System.Collections.Immutable.ImmutableDictionary.ToImmutableDictionary[TSource,TKey,TValue](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 keyComparer, IEqualityComparer`1 valueComparer)
         at System.Collections.Immutable.ImmutableDictionary.ToImmutableDictionary[TSource,TKey,TValue](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector)
         at publisher.Publisher.GetCommitIdFiles(CommitId commitId)
         at publisher.Publisher.RunWithCommitId(CommitId commitId, CancellationToken cancellationToken)
         at publisher.Publisher.Run(CancellationToken cancellationToken)
         at publisher.Publisher.ExecuteAsync(CancellationToken cancellationToken)
         at Microsoft.Extensions.Hosting.Internal.Host.TryExecuteBackgroundServiceAsync(BackgroundService backgroundService)
crit: Microsoft.Extensions.Hosting.Internal.Host[10]
      The HostOptions.BackgroundServiceExceptionBehavior is configured to StopHost. A BackgroundService has thrown an unhandled exception, and the IHost instance is stopping. To avoid this behavior, configure this to Ignore; however the BackgroundService will not be restarted.
      System.ArgumentException: An element with the same key but a different value already exists. Key: 'Put'
         at System.Collections.Immutable.ImmutableDictionary`2.HashBucket.Add(TKey key, TValue value, IEqualityComparer`1 keyOnlyComparer, IEqualityComparer`1 valueComparer, KeyCollisionBehavior behavior, OperationResult& result)
         at System.Collections.Immutable.ImmutableDictionary`2.AddRange(IEnumerable`1 items, MutationInput origin, KeyCollisionBehavior collisionBehavior)
         at System.Collections.Immutable.ImmutableDictionary`2.AddRange(IEnumerable`1 pairs, Boolean avoidToHashMap)
         at System.Collections.Immutable.ImmutableDictionary`2.AddRange(IEnumerable`1 pairs)
         at System.Collections.Immutable.ImmutableDictionary.ToImmutableDictionary[TSource,TKey,TValue](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 keyComparer, IEqualityComparer`1 valueComparer)
         at System.Collections.Immutable.ImmutableDictionary.ToImmutableDictionary[TSource,TKey,TValue](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector)
         at publisher.Publisher.GetCommitIdFiles(CommitId commitId)
         at publisher.Publisher.RunWithCommitId(CommitId commitId, CancellationToken cancellationToken)
         at publisher.Publisher.Run(CancellationToken cancellationToken)
         at publisher.Publisher.ExecuteAsync(CancellationToken cancellationToken)
         at Microsoft.Extensions.Hosting.Internal.Host.TryExecuteBackgroundServiceAsync(BackgroundService backgroundService)

Reproduction Steps

Add a revision to an api in your lower level environment and make that the current revision. Run the extractor, approve and complete the PR and you will see the error in the dev publisher run.

guythetechie commented 1 year ago

@anotherRedbeard - when you have a chance, can you please see if this branch fixes your issue?

anotherRedbeard commented 1 year ago

@guythetechie - I tried that branch and still get an error. It is different than the last error and it seems to indicate it's trying to delete the rev=2 api (because that folder was removed/renamed in the repo) but fails because it is the current revision.

Here is the error:

crit: Publisher[0]
      System.InvalidOperationException: HTTP request to URI https://management.azure.com/subscriptions/***/resourceGroups/***/providers/Microsoft.ApiManagement/service/***/apis/echo-api;rev=2?api-version=2021-12-01-preview failed with status code 400. Content is '{"error":{"code":"ValidationError","message":"Cannot delete the current revision of an API.","details":null}}'.
         at common.HttpPipelineExtensions.Validate(Response response, Uri requestUri)
         at common.HttpPipelineExtensions.DeleteResource(HttpPipeline pipeline, Uri uri, CancellationToken cancellationToken)
         at publisher.Program.<>c__DisplayClass10_0.<<GetDeleteRestResource>b__0>d.MoveNext()
      --- End of stack trace from previous location ---
         at publisher.Api.Delete(ApiName apiName, ServiceUri serviceUri, DeleteRestResource deleteRestResource, ILogger logger, CancellationToken cancellationToken)
         at publisher.Api.ProcessDeletedApi(ApiName apiName, ApiInformationFile deletedApiInformationFile, ApiSpecificationFile deletedSpecificationFile, JsonObject configurationApiJson, ServiceDirectory serviceDirectory, ServiceUri serviceUri, PutRestResource putRestResource, DeleteRestResource deleteRestResource, ILogger logger, CancellationToken cancellationToken)
         at publisher.Api.<>c__DisplayClass0_0.<<ProcessDeletedArtifacts>b__4>d.MoveNext()
      --- End of stack trace from previous location ---
         at System.Threading.Tasks.Parallel.<>c__50`1.<<ForEachAsync>b__50_0>d.MoveNext()
      --- End of stack trace from previous location ---
         at common.IEnumerableExtensions.ForEachParallel[T](IEnumerable`1 enumerable, Func`2 action, CancellationToken cancellationToken)
         at publisher.Api.ProcessDeletedArtifacts(IReadOnlyCollection`1 files, JsonObject configurationJson, ServiceDirectory serviceDirectory, ServiceUri serviceUri, PutRestResource putRestResource, DeleteRestResource deleteRestResource, ILogger logger, CancellationToken cancellationToken)
         at publisher.Service.ProcessDeletedArtifacts(IReadOnlyCollection`1 files, JsonObject configurationJson, ServiceDirectory serviceDirectory, ServiceUri serviceUri, ListRestResources listRestResources, PutRestResource putRestResource, DeleteRestResource deleteRestResource, ILogger logger, CancellationToken cancellationToken)
         at publisher.Publisher.ProcessDeletedCommitIdFiles(IReadOnlyCollection`1 deletedCommitIdFiles, CancellationToken cancellationToken)
         at publisher.Publisher.RunWithCommitId(CommitId commitId, CancellationToken cancellationToken)
         at publisher.Publisher.Run(CancellationToken cancellationToken)
         at publisher.Publisher.ExecuteAsync(CancellationToken cancellationToken)
guythetechie commented 1 year ago

@anotherRedbeard - based on our discussion, I've added a check to ensure the revision is not current prior to deletion. Can you try again with the latest code in the branch?

aristosvo commented 1 year ago

This error also shows up in our environments but for different reasons, I'll check the details and see if this branch fixes it.

waelkdouh commented 1 year ago

@aristosvo can we close this issue?

aristosvo commented 1 year ago

Sure! Thanks a lot:)

MouniR-123 commented 6 months ago

Hi, I'm using the latest release V5.1.4. our api has 3 revisions, when I want to make second revision current, by removing isCurrent: true in the first revision (apiInformation.json.json) and add it in the second revision apiInformation.json.json its not working as expected. Still its keeping reviosn1 active, please suggest how can I set revision2 active?

waelkdouh commented 6 months ago

Yes this is a known issue and will be fixed in the next release.

simonTTIS commented 5 months ago

Hi,

The next major version has been released (v6.0.0-alpha.1.0.4). It is still in alpha but it seems that it is worse than before.

It is not capable to publish additional revisions to an api. For this use case, I am having the error "Can't set new revision as current". And there is no issue, if I created the revision manually in advance. Can you check ?

waelkdouh commented 5 months ago

Please avoid using v6 for now. We are working on fixing some issues with that version. For now please use v5.x.

waelkdouh commented 3 months ago

@simonTTIS we just tested under v6.0.0-rc1 and the issue has been resolved. Please test and let us know.

einzweirad commented 3 months ago

@waelkdouh The publish of new revisions does still not work completely with v6.0.0-rc1. The exception and the error message described above is no longer thrown, but the publisher does not publish the revision set as current in the target APIM.

New revision can be extracted and are also reflecting the setting (isCurrent: true) in the apiInformation. Also the path structure is valid and the other revisions are put in the folder with their revision id. However the publisher does not change the current revision when this is published and keeps the previous revision as the current.