Upload spark libraries to Fabric Environment - Githubissues

Azure-Samples / modern-data-warehouse-dataops

DataOps for Microsoft Data Platform technologies. https://aka.ms/dataops-repo

MIT License

590 stars 462 forks source link

Upload spark libraries to Fabric Environment #872

Open promisinganuj opened 10 hours ago

promisinganuj commented 10 hours ago

As part of the initial setup, we need to upload the python OTEL library and environment.yml files to Fabric Environment.

Reference: https://github.com/Azure-Samples/modern-data-warehouse-dataops/blob/feat/e2e-fabric-dataops-sample/e2e_samples/fabric_dataops_sample/scripts/setup_fabric_environment.py

DoD

[ ] Upload (Stage) the environment file "otel_monitor_invoker.py" to the Fabric Environment.
[ ] Upload (Stage) the "environment.yml" to the Fabric Environment.
[ ] Publish the libraries.
[ ] Do it before enabling the Fabric GIT integration so that the environment gets synced to the repo.

Out of Scope

The same process would also be performed as part of CI/CD process, but that it out of scope of this item.