Enhance the automatic git action scripts on pull into dev and main branch to actually deploy and run a test project

Insight-Services-APAC / APAC-Capability-DAI-DbtFabricSparkNb

MIT License

8 stars 3 forks source link

Enhance the automatic git action scripts on pull into dev and main branch to actually deploy and run a test project #130

Closed grantkriegerai closed 3 months ago

grantkriegerai commented 3 months ago

Enhance the automatic git action scripts on pull into dev and main branch to actually deploy and run a test project. We need to automate more of the testing process as this is consuming time.

Things to note: A Fabric workspace is required for this. Datalakes should have all objects removed prior to starting Fabric Capacity should be stopped on completion A complex project should be used that tests a wide range of features. EG ptfproj in samples

KRodriguez-Insight commented 3 months ago

Update : Service request (Request Raised by Ann) for a Dev workspace will take 3 weeks due to some limitations on capacity. As an alternative we can created a new workspace in another tenant, Test Repo and Service principal has been created, and will continue to test.

KRodriguez-Insight commented 3 months ago

Update on Testing using Service Principal.

Setup and app with relevant permissions, as show below
Created a App Registration group and added to the workspace as Admin (Tried contributor as well)
Inside the Tenant setting, The following permissions for Service Principal was enabled
From the Framework execution side, the Service principal authorization was successful
However, it failed in get workspace id / write operation with the below error. (At the time of upload metadata extract notebook)

Reading from the documentation , understand that access to Fabric API through service principals are limited https://learn.microsoft.com/en-us/fabric/admin/metadata-scanning-enable-read-only-apis cc: @insightgrantkrieger

KRodriguez-Insight commented 3 months ago

Update : Was able to create a account with non MFA (Exempted for 14 days) in own tenant, and was able to deploy and execute the frame work through GIT Actions

This is on a test repository , will test and formalize the change cc: @insightgrantkrieger

KRodriguez-Insight commented 3 months ago

Updated the documentation on the limitations on Service Principal

KRodriguez-Insight commented 3 months ago

The Git action has been created and tested in a separate repo on a test Tenant, However it is setup to run locally now., once the workspace is created, it will be enabled to run without switches.

cc: @insightgrantkrieger