search

microsoft / coe-starter-kit

Other
742 stars 218 forks source link

[CoE Starter Kit - BUG] The crawling results differ using Dataflows or Sync flows in the CoE. #7480

Open madwalter3 opened 8 months ago

madwalter3 commented 8 months ago

Does this bug already exist in our backlog?

Describe the issue

We have made a comparison of the tenant data we get from Dataflows and from the old sync flows in the CoE for all apps (Flows are still in evaluation).

These are our results:

Missing apps:

  1. Component libraries are not synced with Dataflows and thus missing.
  2. Model driven apps in developer environments are not synced with Dataflows and thus missing.
  3. All apps within Teams environments are not synced with the Dataflows and thus missing.
  4. Custom pages that have a Power Apps Release Version older than 3.23071.17.0 are not synced by the Dataflow and thus missing.

Wrong data for apps:

  1. The column "App Modified On" is always filled with the GMT time with the Dataflows and thus wrong. The flows were able to write down the correct time (even summer and winter time has been considered).
  2. The column "App Owner" is not synced correctly for all Model-driven apps with Dataflows. Instead of the real owner it says SYSTEM.
  3. All SharePoint Forms are wrongly listed as Canvas apps in the column "App Type" with Dataflows. You can identify them by filtering the column "SharePoint Form URL" which has only a value when it is a SharePoint Forms.

Could you kindly give us a status on the issues?

Expected Behavior

The Dataflows should not miss any data which the former sync flows were able to.

What solution are you experiencing the issue with?

Core

What solution version are you using?

1.70

What app or flow are you having the issue with?

CoE BYODL Apps

What method are you using to get inventory and telemetry?

Data Export

Steps To Reproduce

No response

Anything else?

No response

AB#2066

Jenefer-Monroe commented 8 months ago

Thank you for testing. Can you please see what version of Core you have installed? image

madwalter3 commented 8 months ago

Hi, we are using version 4.17 image

Jenefer-Monroe commented 8 months ago

Thank you.

Can you please confirm, these things are missing or wrong in the Datalake files themselves yes? That is my understanding but want to be sure I'm not missing anything.

Thanks again

madwalter3 commented 8 months ago

Hi Jenefer-Monroe, yes that is correct :) Thanks for investigation.

Jenefer-Monroe commented 8 months ago

OK thank you for validating, and for accumulating this list. Unfortunately this is feedback for the product team on the Data Export feature rather than on our kit's consumption of it. If you have a good mechanism for product support or a product / Microsoft contact you work with, I suggest you reach out to them to provide this feedback directly. I will also mark it as a product gap on our side and we can press as well.

madwalter3 commented 5 months ago

For those who might be interested I got some replies from the support team:

1. Component libraries --> Won't fix, are stored as Canvas Apps by Authoring or source system and we are not planning to transform any of the properties. Currently we are expanding the properties list, if any other columns are identifiable as component library will send it in the new property. 2. All apps within Teams environments --> This is a bug they are currently working on (as of 03/2024). 3. Custom pages that have a Power Apps Release Version older than 3.23071.17.0 --> They are sending it. Need an example to analyze if any other issues. 4. App Modified On timestamp always GMT --> Won't fix. Since analytics is not the data producer, so analytics service will not do any transformation, so continue to be UTC format. 5. The column "App Owner" wrong for Model-driven apps --> Expansion going on this front. 6. SharePoint Forms are wrongly listed as Canvas apps. --> Won't fix, are stored as Canvas Apps by Authoring or source system and we are planning to transform any of the properties. Currently we are expanding the properties list, if any other columns are identifiable as component library will send it in the new property.