[CoE Starter Kit - BUG] Our tries to avoid running inventory flows automatically after upgrade are failing sometimes

[miroslav-harlas]

Describe the issue

Hello, we have pretty big usage in our tenant, when using Incremental approach and 7 days back as a value in respective environment variable, the inventory takes more than 24 hours (> 1800 environments, > 20k apps, > 40k cloud flows). Due to this situation, we customize schedule of couple of cloud flows and intentionally trigger inventory cloud flows twice a week only instead of having ti running every day. This activity is causing the creation of the unmanaged layer, which we are fully aware of its impact to upgrade process. Due to this situation, we are execute our CoE starter Kit upgrade process in following way:

1. Turn Off all the scheduled inventory cloud flows (in solution)
2. Remove unmanaged layer from all cloud flows
3. Upgrade the CoE starter Kit Core solution to the latest version
4. Configure the schedule of some of the cloud flows, while we expect it is off still
5. Turn specified inventory flows On one-by-one within a period of several days - do this in a controlled way while we are trying avoid to run some of them in parallel - we are ensuring to not overload system and to avoid conflicts of various inventory & clean up flows (e.g. turn on clean up flows only after inventory is completed while ensure that Admin | Sync Template v3 is not triggered again before first run of the cleanup is completed)

The issue, which we are facing with the process above is that sometimes some inventory cloud flows are Turn On immediately after the upgrade process of the solution, and those are being triggered accordingly to the default trigger. In other ways the step 1. of the process described above - "Turn cloud flows off before upgrade" is being ignored by upgrade process. This behavior is random, we have not been facing the same issue during each upgrade, but sometimes only, and the impacted cloud flows are different after each upgrade.

Expected Behavior

We are in a need to know the safe step-by-step process how to upgrade CoE Starter Kit Core solution, while preventing starting some of the cloud flows automatically using the default built in schedule. Our steps, as described above, seems not to be applicable. Thank you in advance for any advice. Miroslav Harlas

What solution are you experiencing the issue with?

Core

What solution version are you using?

4.2

What app or flow are you having the issue with?

Admin | Sync Template v3

Steps To Reproduce

Steps are provided above, but our real-life experience is that the behavior is random, unstable.

Anything else?

No response

[Jenefer-Monroe]

I'm afraid you are hitting some issues with the product upgrade processes outside of our control, and I'm afraid I'm not certain of how to avoid the issue you are referencing.

I think this order may work, switching 2&3. I know that you can remove the unmanaged layer later, and I think this will avoid the auto-trigger.

1. Turn Off all the scheduled inventory cloud flows (in solution)
2. Upgrade the CoE starter Kit Core solution to the latest version
3. Remove unmanaged layer from all cloud flows
4. Configure the schedule of some of the cloud flows, while we expect it is off still
5. Turn specified inventory flows On one-by-one within a period of several days

As an aside, I presume you are already aware of the upcoming Self Service Analytics (aka BYODL) feature. It's created with tenants of your size in mind. We are working hard with the product team to have it ready in the next few quarters, but thought that you might enjoy some pre-reading:

• https://docs.microsoft.com/power-platform/admin/self-service-analytics
• https://docs.microsoft.com/power-platform-release-plan/2021wave1/power-platform-governance-administration/custom-reporting-bring-own-azure-data-lake-power-apps-power-automate

[miroslav-harlas]

Hello Jenefer, first of all, we are fully aware about that you and your team are very busy to enable BYODL approach in starter kit, and we are really looking forward :)

Regarding the upgrade proces and regarding the order of the steps, I'm following the CoE starter Kit documentation described here: https://learn.microsoft.com/en-us/power-platform/guidance/coe/after-setup#remove-unmanaged-layers-from-inventory-flows

It is written explicitly in CoE Starter Kit documentation that "Components with unmanaged layers won't receive updates until you remove that unmanaged layer.".

Based on the CoE Starter Kit documentation I must remove unmanaged layer from all components inside of the solution before proceeding to upgrade of the managed solution.

[Jenefer-Monroe]

That is the recommended path yes. However, the changes are received and inserted, but they are not applied until you remove the unmanaged layer.

It's rather complicated, perhaps this visual will help. Switching the order is safe, I'm not certain it will solve your trigger problem but I think it will.

[Jenefer-Monroe]

closing out as no further action for starter kit team