Azure / azure-libraries-for-net

Azure libraries for .Net
MIT License
378 stars 193 forks source link

[QUERY]How to avoid "Not enough available reserved instance servers to satisfy this request" #609

Open ccic opened 5 years ago

ccic commented 5 years ago

Query/Question I failed to create app plan for "Not enough available reserved instance servers to satisfy this request. Currently 4 instances are available. If you are changing instance size you can reserve up to 4 instances at this moment. If you are increasing instance count then you can add extra 4 instances at this moment. Please get available machines and retry later."

Why is this not a Bug or a feature Request?

Is there any method to avoid it? or is there any method for me to query the current max available instance servers? then I will not ask too many resources than it can provide.

Setup (please complete the following information if applicable):

praries880 commented 5 years ago

@ccic Can you add a code snippet for which you got the above error?

ccic commented 5 years ago
    protected static Task BatchProcess<T>(IList<T> source, Func<T, Task> f, int max)
    {
        var initial = (max >> 1);
        var s = new System.Threading.SemaphoreSlim(initial, max);
        _ = Task.Run(async () =>
        {
            for (int i = initial; i < max; i++)
            {
                await Task.Delay(100);
                s.Release();
            }
        });

        return Task.WhenAll(from item in source
                            select Task.Run(async () =>
                            {
                                await s.WaitAsync();
                                try
                                {
                                    await f(item);
                                }
                                finally
                                {
                                    s.Release();
                                }
                            }));
    }

   protected static async Task CreateAppPlanCoreAsync(
        (IAzure azure,
        string name,
        string region,
        string groupName,
        PricingTier pricingTier,
        Microsoft.Azure.Management.AppService.Fluent.OperatingSystem os) package)
    {
        var funcName = "CreateAppPlanCoreAsync";
        using (var cts = new CancellationTokenSource(TimeSpan.FromHours(1)))
        {
            var iAppPlan = package.azure.AppServices.AppServicePlans.GetByResourceGroup(package.groupName, package.name);
            if (iAppPlan == null)
            {
                await package.azure.AppServices.AppServicePlans
                        .Define(package.name)
                        .WithRegion(package.region)
                        .WithExistingResourceGroup(package.groupName)
                        .WithPricingTier(package.pricingTier)
                        .WithOperatingSystem(package.os)
                        .WithCapacity(_scaleOut) // <== _scaleOut is 5~10, If I remove this, the error will disappear.
                        .CreateAsync(cts.Token);
                Console.WriteLine($"{DateTime.Now.ToString("yyyyMMddHHmmss")} Successfully {funcName} for {package.name}");
            }
            else
            {
                Console.WriteLine($"{DateTime.Now.ToString("yyyyMMddHHmmss")} {funcName} for {package.name} already existed");
            }
        }
    }
    // create app service plans. The default ConcurrentCountOfServicePlan is 5, and it is always successful in my test.
    var packages = (from i in Enumerable.Range(0, _argsOption.WebappCount)
                    select (azure : _azure,
                            name : webappNameList[i],
                            region : _argsOption.Location,
                            groupName : _argsOption.GroupName,
                            pricingTier: targetPricingTier,
                            os : Microsoft.Azure.Management.AppService.Fluent.OperatingSystem.Windows)).ToList();

    await BatchProcess(packages, CreateAppPlan, _argsOption.ConcurrentCountOfServicePlan);
ccic commented 5 years ago

I found if I remove ".WithCapacity(_scaleOut)" when creating app service plan, and update the app service plan after web app was attached, in addition, I add "wait and retry" in my code if the updates fails.

This reduces the failures but not avoid it.

I guess it is because the root cause is resource allocation is slow.

nickzhums commented 4 years ago

@ccic are you still experiencing the issue? thanks