search

Azure / Azure-Orbital-Analytics-Samples

Sample solution that demonstrates how to deploy and analyze spaceborne data using Azure Synapse Analytics
https://aka.ms/synapse-geospatial-analytics
MIT License
30 stars 24 forks source link

Feedback updates #4

Closed senthilkungumaraj closed 2 years ago

senthilkungumaraj commented 2 years ago

• The bash scripts mentioned in the README are not executable -> users will need to run chmod +x on them manually Execute permission added to executable files in deploy directory • The README asks the user to wait until the Batch pool's resizing is done, but as the number of dedicated nodes is set to 0, it will never resize • Before I was able to push the Docker container image to the ACR I had to login using az acr login -n <name of registry> -> This is not specified in the README • configure.sh expects environment.yml to be in the current directory, but the README suggests running the script from outside the deploy folder Added the base-path to use an absolute filepath instead of relative filepath. • I get two errors when running configure.sh: - In Progress o + az batch job create --id custom-vision-model-job --pool-id data-cpu-pool 'NoneType' object has no attribute 'startswith' o + az storage blob upload-batch --destination spark-jobs --account-name synhnsj57hcp --source ../src/transforms/spark-jobs incorrect usage: source must be an existing directory I think this is also related to executing outside the directory, updated to absolute path, lets check if it is fixed now. o I have attached the full output of the command to this mail • package.sh has a similary issue as configure.sh: It expects package.py to be in the current directory but is suggested to be run from outside deploy, result in: python3: can't open file 'package.py': [Errno 2] No such file or directory Updated this file also to use absolute path instead of relative path, now users should be able to execute from anywhere even if they miss instruction. • As a last step, the README mentions to follow the steps in "Deployment Procedures" before running the pipeline. I couldn't find this section anywhere, but I do guess it goes into detail on what to upload (input data as well as config files) to the raw storage account - if not, are you planning to add that information somewhere? Updated README to remove reference to “Deployment Procedures”.

mandarinamdar commented 2 years ago

lets add a comment here that this guid is for contributor role basically granting everything

Lets address this

senthilkungumaraj commented 2 years ago

lets add a comment here that this guid is for contributor role basically granting everything

Thanks, @sushilkm. Good catch. Added the comment to clarify the guid.

senthilkungumaraj commented 2 years ago

should we call the batch.account.role.assignment.bicep directly from the main.bicep making pipelineModule a dependency here would slow down the deployment.

@sushilkm I tried adding it to batch account first. Synapse workspace and Batch Account are being created parallel. Synapse workspace's managed identity is system assigned. So for the RBAC on batch account to work, we need Synapse workspace to complete provisioning before batch account.

Maybe we can look into optimizing this further in v2.