microsoft / coe-starter-kit

Other
754 stars 225 forks source link

[CoE Starter Kit - BUG] ISSUE TITLE - Dataflows are not getting refreshed automatically after the Sep 2024 Upgrade for Test Governance Env. #8987

Open Kumarez007 opened 1 month ago

Kumarez007 commented 1 month ago

Does this bug already exist in our backlog?

Describe the issue

Dataflows are not getting refreshed automatically after the Sep 2024 Upgrade for Test Governance Env. I tried to check the cloudflows and the status is in Off. When I try to turn it On it gives me the below error.

Turn on failed. Flow client error returned with status code "BadRequest" and details "{"error":{"code":"XrmEnvironmentVariableAttributeNotFound","message":"Attribute 'value' was not found for environment variable 'admin_ModelAppDataflowID'."}}". Turn on failed. Flow client error returned with status code "BadRequest" and details "{"error":{"code":"XrmEnvironmentVariableAttributeNotFound","message":"Attribute 'value' was not found for environment variable 'admin_AppDataflowID'."}}". Turn on failed. Flow client error returned with status code "BadRequest" and details "{"error":{"code":"XrmEnvironmentVariableAttributeNotFound","message":"Attribute 'value' was not found for environment variable 'admin_FlowDataflowID'."}}". Turn on failed. Flow client error returned with status code "BadRequest" and details "{"error":{"code":"XrmEnvironmentVariableAttributeNotFound","message":"Attribute 'value' was not found for environment variable 'admin_FlowDataflowID'."}}".  Turn on failed. Flow client error returned with status code "BadRequest" and details "{"error":{"code":"XrmEnvironmentVariableAttributeNotFound","message":"Attribute 'value' was not found for environment variable 'admin_EnvironmentDataflowID'."}}".

Expected Behavior

No response

What solution are you experiencing the issue with?

Governance

What solution version are you using?

4.43

What app or flow are you having the issue with?

Dataflows

What method are you using to get inventory and telemetry?

Cloud flows

Steps To Reproduce

No response

Anything else?

No response

Kumarez007 commented 1 month ago

Issue

Kumarez007 commented 1 month ago

Can I get an update on this please

Jenefer-Monroe commented 1 month ago

Hello unfortunately I am not able to find the screenshots and things needed to help you this week as I am at conference. When I am back next week I can follow up to try and assist

Jenefer-Monroe commented 1 month ago

These env variables are supposed to be filled by the setup wizard. Did you run that?

Kumarez007 commented 1 month ago

Hi Jenefer, Yes I tried there as well but it didn't turn on when I did the upgrade

Kumarez007 commented 1 month ago

Is there a solution to turn it on again now?

Jenefer-Monroe commented 1 month ago

Do you have values for all these env vars? image

As well as this one: image

Kumarez007 commented 1 month ago

image

image

Kumarez007 commented 1 month ago

the dataflow value in use is empty in our environment. Please let me know the next step for the solution.

Jenefer-Monroe commented 1 month ago

In core you will see a flow called DebugBYODL>ConfigureEnvironment

Open it for edit and chose your envt and your Maker dataflow, save, and turn on. image

Go manually trigger the Maker Dataflow image

When the dataflow completes this flow will trigger, open the run and you will find the Dataflow ID for Maker in the trigger output image

While here also validate that the Current Environment one is correct: image

Now do the same for the rest of the dataflows in the following order

When you have all the environment variables filled, be sure to turn back off DebugBYODL>ConfigureEnvironment to avoid it needlessly running. And now you can turn on those flows.

Kumarez007 commented 1 month ago

I have followed the above mentioned steps and the Env variables are still not added and I am not able to turn on the dataflows for automatic refresh. please let me know if anything else we can do.

Jenefer-Monroe commented 1 month ago

you must enter the env vars yourself. That was the way to get the values to enter.

Jenefer-Monroe commented 1 month ago

closing out as no further action for starter kit team

Kumarez007 commented 1 month ago

Hi Jenefer,

Even after adding the value it didn't resolve the issue and also the values gets disappeared again after it was entered manually. Any other way to resolve this issue?

Jenefer-Monroe commented 1 month ago

Please share a screenshot of how you added the env var value. What UX did you use?

Kumarez007 commented 1 month ago

ok thanks... let me share you the screenshot now

Kumarez007 commented 1 month ago

image

This is before adding the variables

Jenefer-Monroe commented 1 month ago

Did you use this app to update the env vars? I noticed you are in edit mode, why? I wonder if you have an unmanaged layer on this page as a result of being in edit mode.

Kumarez007 commented 1 month ago

This is after adding the variables image

Kumarez007 commented 1 month ago

Did you use this app to update the env vars? I noticed you are in edit mode, why? I wonder if you have an unmanaged layer on this page as a result of being in edit mode.

I have opened it to add the variables just now and took the screenshot

Kumarez007 commented 1 month ago

Now I Have started the makers dataflow to refresh manually..

image

Kumarez007 commented 1 month ago

and checked the env variables.. It has vanished. please see the screenshot.

image

Jenefer-Monroe commented 1 month ago

Without running the maker table, please do the following

  1. Update the env vars as you have been as shown here image

  2. Browse away from that page (ex go to bookmarks) and come back. Are they still populated?

My guess is taht you are only seeing the local collection and that the env var is not being populated.

Kumarez007 commented 1 month ago

Hi Jenefer,

I tried adding the env variable againa nd went to bookmark page and came back to env varibales.. it is again vanished.

image

Kumarez007 commented 1 month ago

Will we be able to resolve this during this month Upgrade?

Jenefer-Monroe commented 1 month ago

Yep so if you are Sept release then you must have unmanaged layers present on that page. There was a product change in how to set env vars from canvas and we fixed it for Sept. I suspect you have them as you seem to open the app in edit mode. Please check for unmanaged layers on that page and remove them and then try again. image

If that does not work you can just set them in the product UX itself. Be sure to set Current value not Default. image

Kumarez007 commented 1 month ago

I don't see any unmanaged layers

image

Jenefer-Monroe commented 1 month ago

ok please just go ahead and update the env vars in the product UX itself to unblock yourself.

Kumarez007 commented 1 month ago

Yes did it and it worked and I am able to turn on the flows now as below.

image